Sep 16 13:08:12.194866 ip-10-0-143-82 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 13:08:12.640914 ip-10-0-143-82 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 13:08:12.640914 ip-10-0-143-82 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 13:08:12.640914 ip-10-0-143-82 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 13:08:12.640914 ip-10-0-143-82 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 13:08:12.640914 ip-10-0-143-82 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 13:08:12.643758 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.643678    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 13:08:12.647619 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647605    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647620    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647633    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647637    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647640    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647644    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647647    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647650    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647653    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647657    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647660    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647663    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647666    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647668    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647671    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:08:12.647657 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647673    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647676    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647679    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647681    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647684    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647687    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647689    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647692    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647694    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647699    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647703    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647706    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647709    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647712    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647714    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647717    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647719    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647722    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647724    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:08:12.648026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647727    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647729    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647732    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647734    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647736    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647739    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647741    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647744    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647747    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647750    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647753    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647755    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647758    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647760    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647762    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647766    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647768    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647771    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647773    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647776    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:08:12.648483 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647778    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647781    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647783    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647786    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647788    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647790    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647793    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647796    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647799    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647801    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647803    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647806    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647810    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647813    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647816    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647818    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647821    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647823    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647826    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647829    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:08:12.648989 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647831    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647834    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647837    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647839    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647842    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647845    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647848    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647851    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647856    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647860    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647863    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647866    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647868    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647871    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647873    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647876    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647878    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647881    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647883    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647886    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:08:12.649958 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647889    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647892    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647894    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647897    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.647899    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648565    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648571    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648573    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648576    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648579    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648582    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648585    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648588    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648590    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648608    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648612    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648615    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648618    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648621    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:08:12.650626 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648624    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648629    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648633    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648636    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648639    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648642    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648645    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648647    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648650    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648653    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648655    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648658    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648661    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648664    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648668    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648671    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648675    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648678    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648681    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:08:12.651110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648684    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648687    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648690    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648693    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648695    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648698    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648701    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648704    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648706    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648709    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648712    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648714    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648717    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648721    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648723    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648726    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648729    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648731    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648734    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:08:12.651583 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648736    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648739    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648742    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648744    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648746    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648749    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648751    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648754    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648757    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648759    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648762    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648765    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648768    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648770    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648773    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648775    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648777    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648780    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648782    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648785    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:08:12.652081 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648787    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648790    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648792    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648795    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648798    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648801    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648805    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648807    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648814    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648817    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648819    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648822    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648824    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648827    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648830    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648832    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648835    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648837    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648840    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648842    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:08:12.652576 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648845    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648847    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648850    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648852    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648855    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648858    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.648860    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650850    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650862    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650866    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650869    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650872    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650874    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650877    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650880    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650883    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650885    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650888    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650891    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650893    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:08:12.653084 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650896    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650901    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650904    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650907    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650910    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650913    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650916    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650918    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650921    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650924    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650927    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650929    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650932    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650935    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650938    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650940    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650943    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650945    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650947    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650950    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:08:12.653571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650953    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650956    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650959    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650962    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650965    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650968    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650970    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650973    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650975    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650978    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650980    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650983    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650985    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650988    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650991    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650993    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.650996    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651000    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651003    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651005    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:08:12.654069 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651008    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651010    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651012    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651015    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651017    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651020    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651023    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651025    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651028    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651031    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651033    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651036    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651039    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651042    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651044    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651047    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651049    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651052    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651054    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:08:12.654564 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651058    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651062    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651065    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651068    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651071    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651073    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651076    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651080    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651082    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651084    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651087    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651091    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651095    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651097    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651100    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651103    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651105    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651108    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651111    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:08:12.655098 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651114    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651116    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651119    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651122    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651124    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651127    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651130    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651133    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651195    2556 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651232    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651238    2556 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651242    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651246    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651249    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651253    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651257    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651260    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651263    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651266    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651270    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651273    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651276    2556 flags.go:64] FLAG: --cgroup-root=""
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651279    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 13:08:12.655556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651282    2556 flags.go:64] FLAG: --client-ca-file=""
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651284    2556 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651288    2556 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651292    2556 flags.go:64] FLAG: --cluster-domain=""
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651295    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651298    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651301    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651305    2556 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651309    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651312    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651315    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651318    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651321    2556 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651324    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651327    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651332    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651335    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651339    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651342    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651345    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651348    2556 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651351    2556 flags.go:64] FLAG: --enable-server="true"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651354    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651358    2556 flags.go:64] FLAG: --event-burst="100"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651361    2556 flags.go:64] FLAG: --event-qps="50"
Sep 16 13:08:12.656125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651364    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651367    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651370    2556 flags.go:64] FLAG: --eviction-hard=""
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651374    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651377    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651380    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651383    2556 flags.go:64] FLAG: --eviction-soft=""
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651386    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651389    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651391    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651395    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651398    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651400    2556 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651403    2556 flags.go:64] FLAG: --feature-gates=""
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651407    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651410    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651413    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651416    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651419    2556 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651422    2556 flags.go:64] FLAG: --help="false"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651425    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651428    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651433    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 13:08:12.656753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651436    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651439    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651442    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651445    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651448    2556 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651451    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651454    2556 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651456    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651459    2556 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651462    2556 flags.go:64] FLAG: --kube-reserved=""
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651466    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651469    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651472    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651474    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651477    2556 flags.go:64] FLAG: --lock-file=""
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651480    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651483    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651486    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651491    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651494    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651498    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651501    2556 flags.go:64] FLAG: --logging-format="text"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651504    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651509    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 13:08:12.657346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651511    2556 flags.go:64] FLAG: --manifest-url=""
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651514    2556 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651522    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651525    2556 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651529    2556 flags.go:64] FLAG: --max-pods="110"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651531    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651534    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651538    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651541    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651544    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651547    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651550    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651557    2556 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651560    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651563    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651566    2556 flags.go:64] FLAG: --pod-cidr=""
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651569    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651574    2556 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651578    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651581    2556 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651583    2556 flags.go:64] FLAG: --port="10250"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651586    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651590    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-088530208d75c23a1"
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651607    2556 flags.go:64] FLAG: --qos-reserved=""
Sep 16 13:08:12.657941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651610    2556 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651614    2556 flags.go:64] FLAG: --register-node="true"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651616    2556 flags.go:64] FLAG: --register-with-taints=""
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651620    2556 flags.go:64] FLAG: --registry-burst="10"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651623    2556 flags.go:64] FLAG: --registry-qps="5"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651626    2556 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651629    2556 flags.go:64] FLAG: --reserved-memory=""
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651635    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651638    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651641    2556 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651644    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651647    2556 flags.go:64] FLAG: --runonce="false"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651650    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651653    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651656    2556 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651659    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651663    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651666    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651669    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651671    2556 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651675    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651678    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651681    2556 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651684    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651687    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 13:08:12.658570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651690    2556 flags.go:64] FLAG: --system-cgroups=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651693    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651698    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651700    2556 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651703    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651708    2556 flags.go:64] FLAG: --tls-min-version=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651710    2556 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651713    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651716    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651719    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651721    2556 flags.go:64] FLAG: --v="2"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651725    2556 flags.go:64] FLAG: --version="false"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651729    2556 flags.go:64] FLAG: --vmodule=""
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651733    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.651738    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651846    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651849    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651853    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651856    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651858    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651861    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651864    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651866    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:08:12.659299 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651870    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651873    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651876    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651879    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651881    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651884    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651887    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651889    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651892    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651895    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651897    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651900    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651902    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651905    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651907    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651910    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651912    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651915    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651917    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:08:12.659886 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651920    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651922    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651925    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651928    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651932    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651935    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651938    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651940    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651943    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651947    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651950    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651954    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651957    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651961    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651964    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651967    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651970    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651972    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651975    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651977    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:08:12.660362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651980    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651983    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651987    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651990    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651993    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651996    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.651999    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652002    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652005    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652007    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652010    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652012    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652015    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652017    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652020    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652023    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652027    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652030    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652032    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652035    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:08:12.660873 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652037    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652040    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652042    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652045    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652048    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652052    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652055    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652057    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652060    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652062    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652065    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652068    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652070    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652073    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652075    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652078    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652080    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652083    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652086    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652088    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:08:12.661352 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652091    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652094    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652098    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652101    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652104    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652107    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652110    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652112    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652133    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652139    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652144    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.652149    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.652915    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.660416    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.660431    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 13:08:12.661857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.660450    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.660456    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660511    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660516    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660519    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660522    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660525    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660528    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660531    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660534    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660536    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660539    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660542    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660544    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660547    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660549    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660552    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660555    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660557    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:08:12.662277 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660559    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660562    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660566    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660570    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660573    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660575    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660578    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660581    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660584    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660586    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660589    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660591    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660608    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660611    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660613    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660616    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660619    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660622    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660625    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660627    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:08:12.662819 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660630    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660633    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660635    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660638    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660641    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660643    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660646    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660649    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660651    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660654    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660657    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660660    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660662    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660665    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660667    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660670    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660673    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660675    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660678    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660681    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:08:12.663304 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660684    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660686    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660689    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660692    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660695    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660698    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660703    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660706    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660709    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660712    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660715    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660718    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660722    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660725    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660728    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660731    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660734    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660737    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660740    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:08:12.663860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660743    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660745    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660748    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660751    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660754    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660757    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660759    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660762    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660765    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660767    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660770    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660773    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660775    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660778    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660781    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660784    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660787    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660790    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660792    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660795    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:08:12.664335 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660798    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660801    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660804    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.660809    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660930    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660936    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660939    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660942    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660945    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660948    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660951    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660954    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660957    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660959    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660962    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:08:12.664856 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660964    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660968    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660971    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660973    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660976    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660979    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660981    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660984    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660986    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660989    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660993    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.660997    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661000    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661003    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661005    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661008    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661011    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661014    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661017    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:08:12.665228 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661019    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661022    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661024    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661027    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661030    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661033    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661035    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661038    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661041    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661043    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661045    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661048    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661051    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661053    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661056    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661058    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661061    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661064    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661066    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661069    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:08:12.665690 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661071    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661074    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661077    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661079    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661082    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661084    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661088    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661091    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661094    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661096    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661099    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661102    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661105    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661107    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661110    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661113    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661115    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661118    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661120    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661122    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:08:12.666171 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661125    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661127    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661130    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661132    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661135    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661138    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661140    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661143    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661146    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661148    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661151    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661153    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661155    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661158    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661161    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661165    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661168    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661171    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661174    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:08:12.666698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661177    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661180    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661183    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661186    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661188    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661193    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661196    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661199    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661201    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:12.661204    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.661208    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.661298    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.663568    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.664623    2556 server.go:1019] "Starting client certificate rotation"
Sep 16 13:08:12.667159 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.664723    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 13:08:12.667534 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.664748    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 13:08:12.668319 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.668306    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 13:08:12.669481 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.669463    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 13:08:12.680754 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.680736    2556 log.go:25] "Validated CRI v1 runtime API"
Sep 16 13:08:12.684658 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.684639    2556 log.go:25] "Validated CRI v1 image API"
Sep 16 13:08:12.685555 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.685540    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 13:08:12.688853 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.688834    2556 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b47f1f96-3b04-4e59-9ddb-e6ad2ac70323:/dev/nvme0n1p4 ea0e1554-00b5-4806-9afa-316b9b3534b4:/dev/nvme0n1p3]
Sep 16 13:08:12.688902 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.688853    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 13:08:12.693544 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.693527    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 13:08:12.694838 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.694726    2556 manager.go:216] Machine: {Timestamp:2026-09-16 13:08:12.69330185 +0000 UTC m=+0.370016276 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099147 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26d1538809a97e5cb5db317e7984ae SystemUUID:ec26d153-8809-a97e-5cb5-db317e7984ae BootID:77a4266b-e2d3-4a3e-bb6a-00622f6cb3f8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c4:d5:34:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c4:d5:34:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:fe:7f:da:cf:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 13:08:12.694838 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.694833    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 13:08:12.694932 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.694904    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 13:08:12.698313 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.698291    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 13:08:12.698438 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.698316    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-82.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 13:08:12.699133 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.699122    2556 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 13:08:12.699171 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.699135    2556 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 13:08:12.699171 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.699149    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 13:08:12.699171 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.699162    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 13:08:12.700294 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.700285    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 13:08:12.704635 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704624    2556 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 13:08:12.704748 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704739    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 13:08:12.704789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704767    2556 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 13:08:12.704789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704777    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 13:08:12.704789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704788    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 13:08:12.704897 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704797    2556 kubelet.go:398] "Adding apiserver pod source"
Sep 16 13:08:12.704897 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.704806    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 13:08:12.705091 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.705080    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 13:08:12.705124 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.705099    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 13:08:12.707735 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.707719    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 13:08:12.709842 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.709829    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 13:08:12.709906 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.709858    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 13:08:12.711965 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.711953    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.711970    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.711975    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.711981    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.711990    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.711995    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712008    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712014    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 13:08:12.712020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712021    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 13:08:12.712240 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712027    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 13:08:12.712240 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712036    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 13:08:12.712240 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712042    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 13:08:12.712941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712931    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 13:08:12.712941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.712939    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 13:08:12.717053 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.717035    2556 server.go:1268] "Started kubelet"
Sep 16 13:08:12.717153 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.717102    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 13:08:12.717208 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.717159    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 13:08:12.717255 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.717233    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 13:08:12.717940 ip-10-0-143-82 systemd[1]: Started Kubernetes Kubelet.
Sep 16 13:08:12.719048 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.717994    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-82.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 13:08:12.719048 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.718113    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-82.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 13:08:12.719407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.719385    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 13:08:12.720039 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.720014    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 13:08:12.721100 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.721079    2556 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 13:08:12.725079 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.725061    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 13:08:12.725547 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.725529    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 13:08:12.726238 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.726220    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 13:08:12.726419 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.726401    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:12.726992 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.726970    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 13:08:12.727068 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.726998    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 13:08:12.727068 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.726978    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 13:08:12.727158 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727079    2556 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 13:08:12.727158 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727087    2556 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 13:08:12.727252 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727180    2556 factory.go:55] Registering systemd factory
Sep 16 13:08:12.727252 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727196    2556 factory.go:223] Registration of the systemd container factory successfully
Sep 16 13:08:12.727252 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.725812    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-82.ec2.internal.18d5cf0f9130b4c4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-82.ec2.internal,UID:ip-10-0-143-82.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-82.ec2.internal,},FirstTimestamp:2026-09-16 13:08:12.716987588 +0000 UTC m=+0.393702017,LastTimestamp:2026-09-16 13:08:12.716987588 +0000 UTC m=+0.393702017,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-82.ec2.internal,}"
Sep 16 13:08:12.727431 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727416    2556 factory.go:153] Registering CRI-O factory
Sep 16 13:08:12.727499 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727434    2556 factory.go:223] Registration of the crio container factory successfully
Sep 16 13:08:12.727499 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727482    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 13:08:12.727499 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727498    2556 factory.go:103] Registering Raw factory
Sep 16 13:08:12.727663 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727513    2556 manager.go:1196] Started watching for new ooms in manager
Sep 16 13:08:12.727894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.727882    2556 manager.go:349] Starting recovery of all containers
Sep 16 13:08:12.730942 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.730917    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-82.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 13:08:12.731037 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.731001    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 13:08:12.736957 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.736938    2556 manager.go:354] Recovery completed
Sep 16 13:08:12.741531 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.741374    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.743770 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.743743    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.743831 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.743786    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.743831 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.743799    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.744389 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.744370    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 13:08:12.744389 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.744388    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 13:08:12.744502 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.744404    2556 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 13:08:12.744502 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.744380    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.745257 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.745234    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.745320 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.745266    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.745320 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.745283    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.745694 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.745683    2556 policy_none.go:49] "None policy: Start"
Sep 16 13:08:12.745730 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.745699    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 13:08:12.745730 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.745709    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 13:08:12.746101 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.746026    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-82.ec2.internal.18d5cf0f92c95f7b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-82.ec2.internal,UID:ip-10-0-143-82.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-82.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-82.ec2.internal,},FirstTimestamp:2026-09-16 13:08:12.743769979 +0000 UTC m=+0.420484406,LastTimestamp:2026-09-16 13:08:12.743769979 +0000 UTC m=+0.420484406,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-82.ec2.internal,}"
Sep 16 13:08:12.746954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.746942    2556 policy_none.go:47] "Start"
Sep 16 13:08:12.749771 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.749748    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5rms5"
Sep 16 13:08:12.754126 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.754036    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-82.ec2.internal.18d5cf0f92c9b6ca  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-82.ec2.internal,UID:ip-10-0-143-82.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-82.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-82.ec2.internal,},FirstTimestamp:2026-09-16 13:08:12.74379233 +0000 UTC m=+0.420506757,LastTimestamp:2026-09-16 13:08:12.74379233 +0000 UTC m=+0.420506757,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-82.ec2.internal,}"
Sep 16 13:08:12.755434 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.755416    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5rms5"
Sep 16 13:08:12.756850 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.756800    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-82.ec2.internal.18d5cf0f92c9e697  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-82.ec2.internal,UID:ip-10-0-143-82.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-82.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-82.ec2.internal,},FirstTimestamp:2026-09-16 13:08:12.743804567 +0000 UTC m=+0.420518994,LastTimestamp:2026-09-16 13:08:12.743804567 +0000 UTC m=+0.420518994,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-82.ec2.internal,}"
Sep 16 13:08:12.780303 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780279    2556 manager.go:337] "Starting Device Plugin manager"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.780338    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780352    2556 server.go:87] "Starting device plugin registration server"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780749    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780760    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780861    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780950    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.780958    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.781503    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.781544    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.792395    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.793578    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.793638    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.793663    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.793735    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 13:08:12.803690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.795668    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:08:12.881435 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.881400    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.882365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.882347    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.882457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.882376    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.882457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.882385    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.882457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.882407    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.892936 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.892877    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.892936 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.892899    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-82.ec2.internal\": node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:12.893842 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.893822    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"]
Sep 16 13:08:12.893908 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.893896    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.895339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.895322    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.895416 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.895350    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.895416 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.895366    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.897631 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.897615    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.897707 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.897683    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.897750 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.897710    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.898276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.898255    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.898276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.898279    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.898384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.898288    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.898384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.898259    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.898384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.898346    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.898384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.898355    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.900415 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.900402    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.900469 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.900427    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:12.901040 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.901027    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:12.901088 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.901054    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:12.901088 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.901068    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:12.920118 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.920101    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:12.927294 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.927271    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/32a26a83e8371e1e188502dccce97eac-config\") pod \"kube-apiserver-proxy-ip-10-0-143-82.ec2.internal\" (UID: \"32a26a83e8371e1e188502dccce97eac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.927371 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.927296    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df32444e188d3dde6bc7b731c17e8041-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal\" (UID: \"df32444e188d3dde6bc7b731c17e8041\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.927371 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:12.927315    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df32444e188d3dde6bc7b731c17e8041-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal\" (UID: \"df32444e188d3dde6bc7b731c17e8041\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.936238 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.936219    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-82.ec2.internal\" not found" node="ip-10-0-143-82.ec2.internal"
Sep 16 13:08:12.940679 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:12.940662    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-82.ec2.internal\" not found" node="ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.020998 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.020972    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.028230 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.028214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/32a26a83e8371e1e188502dccce97eac-config\") pod \"kube-apiserver-proxy-ip-10-0-143-82.ec2.internal\" (UID: \"32a26a83e8371e1e188502dccce97eac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.028321 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.028300    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df32444e188d3dde6bc7b731c17e8041-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal\" (UID: \"df32444e188d3dde6bc7b731c17e8041\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.028359 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.028313    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df32444e188d3dde6bc7b731c17e8041-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal\" (UID: \"df32444e188d3dde6bc7b731c17e8041\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.028359 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.028326    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/32a26a83e8371e1e188502dccce97eac-config\") pod \"kube-apiserver-proxy-ip-10-0-143-82.ec2.internal\" (UID: \"32a26a83e8371e1e188502dccce97eac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.028359 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.028345    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df32444e188d3dde6bc7b731c17e8041-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal\" (UID: \"df32444e188d3dde6bc7b731c17e8041\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.028451 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.028381    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df32444e188d3dde6bc7b731c17e8041-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal\" (UID: \"df32444e188d3dde6bc7b731c17e8041\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.121427 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.121397    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.222148 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.222077    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.239522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.239506    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.244755 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.244739    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:13.323101 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.323067    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.423593 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.423569    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.524076 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.523997    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.624504 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.624471    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.664792 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.664763    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 13:08:13.665201 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.664900    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 13:08:13.724870 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.724847    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.760346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.760316    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 13:03:13 +0000 UTC" deadline="2028-07-03 11:35:39.770970238 +0000 UTC"
Sep 16 13:08:13.760346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.760344    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15742h27m26.010629166s"
Sep 16 13:08:13.815215 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.815161    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:08:13.816166 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.816149    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:08:13.816263 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.816191    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:08:13.816263 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.816207    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:08:13.825433 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.825397    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.851373 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.851349    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:08:13.883904 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:13.883865    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf32444e188d3dde6bc7b731c17e8041.slice/crio-da909d86479af870d892a47f3dc3da014ede0f1c2bb2a9580499cc8f5a478161 WatchSource:0}: Error finding container da909d86479af870d892a47f3dc3da014ede0f1c2bb2a9580499cc8f5a478161: Status 404 returned error can't find the container with id da909d86479af870d892a47f3dc3da014ede0f1c2bb2a9580499cc8f5a478161
Sep 16 13:08:13.884661 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:13.884643    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a26a83e8371e1e188502dccce97eac.slice/crio-013dbc90f45c1e357f9f8b894f78e8da96b374dc15eba810647a654e8ce4d794 WatchSource:0}: Error finding container 013dbc90f45c1e357f9f8b894f78e8da96b374dc15eba810647a654e8ce4d794: Status 404 returned error can't find the container with id 013dbc90f45c1e357f9f8b894f78e8da96b374dc15eba810647a654e8ce4d794
Sep 16 13:08:13.888995 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.888877    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:08:13.926152 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:13.926123    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-82.ec2.internal\" not found"
Sep 16 13:08:13.986799 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:13.986778    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:08:14.028607 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.028569    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:14.037561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.037546    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 13:08:14.039889 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.039877    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal"
Sep 16 13:08:14.052882 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.052866    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 13:08:14.066881 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.066823    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:08:14.705840 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.705806    2556 apiserver.go:52] "Watching apiserver"
Sep 16 13:08:14.712592 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.712560    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 13:08:14.713020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.712993    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-2n6bp","kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv","openshift-cluster-node-tuning-operator/tuned-9pvx5","openshift-dns/node-resolver-td4wf","openshift-image-registry/node-ca-rfr7l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal","openshift-multus/multus-additional-cni-plugins-jh4g2","kube-system/konnectivity-agent-nj4sb","openshift-multus/multus-lhc2r","openshift-multus/network-metrics-daemon-jdkss","openshift-network-diagnostics/network-check-target-jtk45","openshift-network-operator/iptables-alerter-m2k9b","openshift-ovn-kubernetes/ovnkube-node-4rsvv"]
Sep 16 13:08:14.715818 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.715795    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.718361 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718198    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 13:08:14.718361 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718220    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.718361 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718275    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.718586 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718371    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.718586 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718503    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 13:08:14.718900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718877    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 13:08:14.719000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.718933    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h2mtf\""
Sep 16 13:08:14.720744 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.720708    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 13:08:14.720744 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.720729    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.720909 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.720801    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.720967 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.720946    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.721232 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.721198    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 13:08:14.721232 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.721225    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 13:08:14.721362 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.721240    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7v5fn\""
Sep 16 13:08:14.721362 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.721230    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 13:08:14.723106 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.723088    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.723327 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.723305    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nnp7v\""
Sep 16 13:08:14.723910 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.723888    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 13:08:14.724004 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.723959    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.724082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.724033    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:14.724150 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.724098    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:14.726317 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.726290    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 13:08:14.726552 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.726531    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:14.726648 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.726631    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:14.729106 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.729085    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.731307 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.731283    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 13:08:14.731405 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.731287    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4mcv2\""
Sep 16 13:08:14.731683 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.731654    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.731792 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.731665    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.731855 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.731795    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.734061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.734042    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x882g\""
Sep 16 13:08:14.735151 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.735111    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.736580 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.735514    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 13:08:14.738067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738039    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 13:08:14.738371 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738350    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.738557 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738533    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rjjxt\""
Sep 16 13:08:14.738651 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738573    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llhlt\" (UniqueName: \"kubernetes.io/projected/c4c5e114-51f5-4670-85af-f57c14bc6c58-kube-api-access-llhlt\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:14.738651 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a2b4109-e4a9-467b-8ec7-a71ca968b738-host\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.738651 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738635    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwtb2\" (UniqueName: \"kubernetes.io/projected/3a2b4109-e4a9-467b-8ec7-a71ca968b738-kube-api-access-rwtb2\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.738852 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738654    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cnibin\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.738852 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738795    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-var-lib-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.738852 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738828    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-ovnkube-config\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738864    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce94dfbb-3a08-455c-814a-ce6009ae8682-host-slash\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.739026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738892    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-kubelet\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738944    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-ovn\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738970    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:14.739026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.738992    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.739026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739024    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-env-overrides\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739043    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739050    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-ovnkube-script-lib\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-etc-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739139    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-log-socket\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739169    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-cni-netd\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739202    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r29k9\" (UniqueName: \"kubernetes.io/projected/ce94dfbb-3a08-455c-814a-ce6009ae8682-kube-api-access-r29k9\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.739287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739228    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:14.739516 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739335    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.739516 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739379    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.739516 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739407    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-systemd\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739663 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739488    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-cni-bin\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.739663 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.739587    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkgjt\" (UniqueName: \"kubernetes.io/projected/72d9e406-a876-4bcf-b27f-6866da5bf255-kube-api-access-qkgjt\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740231 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740212    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.740374 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740349    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-system-cni-dir\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.740426 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-run-netns\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740429    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce94dfbb-3a08-455c-814a-ce6009ae8682-iptables-alerter-script\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.740471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740453    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-systemd-units\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740560 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-slash\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740560 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740502    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740560 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740524    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-node-log\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740568    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740618    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a2b4109-e4a9-467b-8ec7-a71ca968b738-serviceca\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.740779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.740779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740698    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m66nw\" (UniqueName: \"kubernetes.io/projected/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-kube-api-access-m66nw\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.740779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740726    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.740779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.740752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d9e406-a876-4bcf-b27f-6866da5bf255-ovn-node-metrics-cert\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.742393 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.742373    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 13:08:14.742506 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.742434    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-s7426\""
Sep 16 13:08:14.742778 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.742759    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 13:08:14.742999 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.742763    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.745080 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.745047    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m768p\""
Sep 16 13:08:14.745211 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.745083    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.745407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.745388    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.745538 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.745518    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.745658 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.745590    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:14.748065 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.748045    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.750351 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.750262    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 13:08:14.750351 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.750271    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nxxbq\""
Sep 16 13:08:14.750590 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.750546    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 13:08:14.750687 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.750590    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 13:08:14.767411 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.767388    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ktzxl"
Sep 16 13:08:14.776195 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.776175    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ktzxl"
Sep 16 13:08:14.797975 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.797940    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal" event={"ID":"df32444e188d3dde6bc7b731c17e8041","Type":"ContainerStarted","Data":"da909d86479af870d892a47f3dc3da014ede0f1c2bb2a9580499cc8f5a478161"}
Sep 16 13:08:14.799049 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.799030    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal" event={"ID":"32a26a83e8371e1e188502dccce97eac","Type":"ContainerStarted","Data":"013dbc90f45c1e357f9f8b894f78e8da96b374dc15eba810647a654e8ce4d794"}
Sep 16 13:08:14.827833 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.827809    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842194    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-cni-netd\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842285    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/130b08a5-9b01-4853-9112-f2b2847e61c3-cni-binary-copy\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842314    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-cni-multus\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842348    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz7xd\" (UniqueName: \"kubernetes.io/projected/0a4bdde9-8ef0-4089-9fdf-9414452b6494-kube-api-access-sz7xd\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842379    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842412    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-sys-fs\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842445    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842472    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-cni-bin\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkgjt\" (UniqueName: \"kubernetes.io/projected/72d9e406-a876-4bcf-b27f-6866da5bf255-kube-api-access-qkgjt\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842533    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-hostroot\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842565    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-systemd\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842595    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-host\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-system-cni-dir\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842708    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-run-netns\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.844422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842740    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce94dfbb-3a08-455c-814a-ce6009ae8682-iptables-alerter-script\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842773    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pz5\" (UniqueName: \"kubernetes.io/projected/130b08a5-9b01-4853-9112-f2b2847e61c3-kube-api-access-m6pz5\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-run\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842829    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnd87\" (UniqueName: \"kubernetes.io/projected/834e4aa4-4e23-4dc1-87a1-c8043a851e61-kube-api-access-dnd87\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842870    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/694264fb-023d-4197-b834-514ec068625c-kubelet-config\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842901    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-node-log\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842935    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842960    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysctl-conf\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.842992    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/834e4aa4-4e23-4dc1-87a1-c8043a851e61-tmp\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843022    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-device-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzqh6\" (UniqueName: \"kubernetes.io/projected/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-kube-api-access-hzqh6\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843118    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a2b4109-e4a9-467b-8ec7-a71ca968b738-serviceca\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843150    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m66nw\" (UniqueName: \"kubernetes.io/projected/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-kube-api-access-m66nw\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843184    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843218    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-netns\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-kubernetes\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.845390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843280    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3229d434-09bf-404f-b75a-bbb01395351e-agent-certs\") pod \"konnectivity-agent-nj4sb\" (UID: \"3229d434-09bf-404f-b75a-bbb01395351e\") " pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843306    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a4bdde9-8ef0-4089-9fdf-9414452b6494-hosts-file\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843356    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-var-lib-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-ovnkube-config\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce94dfbb-3a08-455c-814a-ce6009ae8682-host-slash\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843447    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843479    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-tuned\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843506    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3229d434-09bf-404f-b75a-bbb01395351e-konnectivity-ca\") pod \"konnectivity-agent-nj4sb\" (UID: \"3229d434-09bf-404f-b75a-bbb01395351e\") " pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843538    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-kubelet\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-ovn\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843608    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-daemon-config\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843667    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-multus-certs\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843745    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-env-overrides\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-ovnkube-script-lib\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843829    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-kubelet\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843860    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-etc-kubernetes\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.846169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843897    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-etc-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.843928    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-log-socket\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844035    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r29k9\" (UniqueName: \"kubernetes.io/projected/ce94dfbb-3a08-455c-814a-ce6009ae8682-kube-api-access-r29k9\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844070    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysconfig\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-sys\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844149    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844183    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-systemd\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844336    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-modprobe-d\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844453    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-cni-netd\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844763    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-node-log\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844806    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844841    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a2b4109-e4a9-467b-8ec7-a71ca968b738-serviceca\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844904    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/694264fb-023d-4197-b834-514ec068625c-dbus\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844931    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-system-cni-dir\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.844961    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-etc-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845001    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-ovn\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845003    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-log-socket\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.846900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845045    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-systemd\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-run-netns\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845111    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845155    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-registration-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845207    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-systemd-units\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845192    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845249    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-slash\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845312    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-var-lib-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845504    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-cni-bin\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845722    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-ovnkube-config\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845722    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce94dfbb-3a08-455c-814a-ce6009ae8682-iptables-alerter-script\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845804    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845829    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-ovnkube-script-lib\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845848    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-cnibin\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce94dfbb-3a08-455c-814a-ce6009ae8682-host-slash\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.845917    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845923    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-systemd-units\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845946    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d9e406-a876-4bcf-b27f-6866da5bf255-env-overrides\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.847773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845965    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-kubelet\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846384    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846384    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.845914    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846448    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-host-slash\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.846452    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:15.346422688 +0000 UTC m=+3.023137143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d9e406-a876-4bcf-b27f-6866da5bf255-run-openvswitch\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846581    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d9e406-a876-4bcf-b27f-6866da5bf255-ovn-node-metrics-cert\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846800    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llhlt\" (UniqueName: \"kubernetes.io/projected/c4c5e114-51f5-4670-85af-f57c14bc6c58-kube-api-access-llhlt\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-socket-dir-parent\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-cni-bin\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846897    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846964    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-conf-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.846994    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysctl-d\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847028    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-lib-modules\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a2b4109-e4a9-467b-8ec7-a71ca968b738-host\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.848338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847103    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtb2\" (UniqueName: \"kubernetes.io/projected/3a2b4109-e4a9-467b-8ec7-a71ca968b738-kube-api-access-rwtb2\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cnibin\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a2b4109-e4a9-467b-8ec7-a71ca968b738-host\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847226    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-cnibin\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847247    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-system-cni-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847295    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-var-lib-kubelet\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847350    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4bdde9-8ef0-4089-9fdf-9414452b6494-tmp-dir\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847359    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847381    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-socket-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847405    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-cni-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-k8s-cni-cncf-io\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.848900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.847490    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.850699 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.850681    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d9e406-a876-4bcf-b27f-6866da5bf255-ovn-node-metrics-cert\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.853465 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.853442    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r29k9\" (UniqueName: \"kubernetes.io/projected/ce94dfbb-3a08-455c-814a-ce6009ae8682-kube-api-access-r29k9\") pod \"iptables-alerter-m2k9b\" (UID: \"ce94dfbb-3a08-455c-814a-ce6009ae8682\") " pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:14.854267 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.854244    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkgjt\" (UniqueName: \"kubernetes.io/projected/72d9e406-a876-4bcf-b27f-6866da5bf255-kube-api-access-qkgjt\") pod \"ovnkube-node-4rsvv\" (UID: \"72d9e406-a876-4bcf-b27f-6866da5bf255\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:14.856208 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.856184    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:14.856291 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.856213    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:14.856291 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.856225    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:14.856291 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.856283    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:08:15.356263431 +0000 UTC m=+3.032977851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:14.858209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.858187    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m66nw\" (UniqueName: \"kubernetes.io/projected/e53b8f25-b9fe-42b5-8069-ba6fa087d69c-kube-api-access-m66nw\") pod \"multus-additional-cni-plugins-jh4g2\" (UID: \"e53b8f25-b9fe-42b5-8069-ba6fa087d69c\") " pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:14.858398 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.858379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llhlt\" (UniqueName: \"kubernetes.io/projected/c4c5e114-51f5-4670-85af-f57c14bc6c58-kube-api-access-llhlt\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:14.858842 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.858818    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwtb2\" (UniqueName: \"kubernetes.io/projected/3a2b4109-e4a9-467b-8ec7-a71ca968b738-kube-api-access-rwtb2\") pod \"node-ca-rfr7l\" (UID: \"3a2b4109-e4a9-467b-8ec7-a71ca968b738\") " pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:14.947773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947743    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/130b08a5-9b01-4853-9112-f2b2847e61c3-cni-binary-copy\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.947773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947775    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-cni-multus\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947793    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz7xd\" (UniqueName: \"kubernetes.io/projected/0a4bdde9-8ef0-4089-9fdf-9414452b6494-kube-api-access-sz7xd\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947808    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947824    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-sys-fs\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-hostroot\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947878    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-systemd\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947879    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-cni-multus\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947900    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-host\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947944    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-hostroot\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pz5\" (UniqueName: \"kubernetes.io/projected/130b08a5-9b01-4853-9112-f2b2847e61c3-kube-api-access-m6pz5\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947960    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.947987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-run\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948022    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnd87\" (UniqueName: \"kubernetes.io/projected/834e4aa4-4e23-4dc1-87a1-c8043a851e61-kube-api-access-dnd87\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948031    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-sys-fs\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948047    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/694264fb-023d-4197-b834-514ec068625c-kubelet-config\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-run\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysctl-conf\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/834e4aa4-4e23-4dc1-87a1-c8043a851e61-tmp\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-systemd\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948168    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-device-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/694264fb-023d-4197-b834-514ec068625c-kubelet-config\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948197    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzqh6\" (UniqueName: \"kubernetes.io/projected/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-kube-api-access-hzqh6\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948208    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-netns\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-kubernetes\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948302    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3229d434-09bf-404f-b75a-bbb01395351e-agent-certs\") pod \"konnectivity-agent-nj4sb\" (UID: \"3229d434-09bf-404f-b75a-bbb01395351e\") " pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948332    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a4bdde9-8ef0-4089-9fdf-9414452b6494-hosts-file\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948132    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-host\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.948610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948382    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-tuned\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948401    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-device-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3229d434-09bf-404f-b75a-bbb01395351e-konnectivity-ca\") pod \"konnectivity-agent-nj4sb\" (UID: \"3229d434-09bf-404f-b75a-bbb01395351e\") " pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948463    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-daemon-config\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948467    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/130b08a5-9b01-4853-9112-f2b2847e61c3-cni-binary-copy\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948517    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-multus-certs\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948530    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysctl-conf\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-multus-certs\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948570    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-kubelet\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948589    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a4bdde9-8ef0-4089-9fdf-9414452b6494-hosts-file\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948599    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-netns\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948609    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-etc-kubernetes\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948630    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-kubelet\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948697    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysconfig\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948688    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-kubernetes\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948691    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-etc-kubernetes\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948720    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-sys\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948755    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysconfig\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.949483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948755    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-modprobe-d\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/694264fb-023d-4197-b834-514ec068625c-dbus\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-registration-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948816    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-modprobe-d\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948821    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-cnibin\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948853    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-socket-dir-parent\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948858    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/694264fb-023d-4197-b834-514ec068625c-dbus\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948876    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-cni-bin\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948896    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-registration-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948898    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-conf-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-conf-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948934    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysctl-d\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948960    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-lib-modules\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948969    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-var-lib-cni-bin\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948991    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-system-cni-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-var-lib-kubelet\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949045    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4bdde9-8ef0-4089-9fdf-9414452b6494-tmp-dir\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949064    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-lib-modules\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949075    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-socket-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949065    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-sysctl-d\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-socket-dir-parent\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-cni-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949109    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-var-lib-kubelet\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.948992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-cnibin\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-k8s-cni-cncf-io\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949168    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-host-run-k8s-cni-cncf-io\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949160    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.949277    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-socket-dir\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:14.949330    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:08:15.44931149 +0000 UTC m=+3.126025923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949344    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-cni-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949351    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3229d434-09bf-404f-b75a-bbb01395351e-konnectivity-ca\") pod \"konnectivity-agent-nj4sb\" (UID: \"3229d434-09bf-404f-b75a-bbb01395351e\") " pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949378    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/130b08a5-9b01-4853-9112-f2b2847e61c3-multus-daemon-config\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949384    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4bdde9-8ef0-4089-9fdf-9414452b6494-tmp-dir\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949421    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834e4aa4-4e23-4dc1-87a1-c8043a851e61-sys\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.950890 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.949433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/130b08a5-9b01-4853-9112-f2b2847e61c3-system-cni-dir\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.951661 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.951634    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/834e4aa4-4e23-4dc1-87a1-c8043a851e61-tmp\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.951775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.951696    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/834e4aa4-4e23-4dc1-87a1-c8043a851e61-etc-tuned\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.951775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.951708    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3229d434-09bf-404f-b75a-bbb01395351e-agent-certs\") pod \"konnectivity-agent-nj4sb\" (UID: \"3229d434-09bf-404f-b75a-bbb01395351e\") " pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:14.956532 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.956469    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzqh6\" (UniqueName: \"kubernetes.io/projected/e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178-kube-api-access-hzqh6\") pod \"aws-ebs-csi-driver-node-jnntv\" (UID: \"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:14.956925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.956904    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnd87\" (UniqueName: \"kubernetes.io/projected/834e4aa4-4e23-4dc1-87a1-c8043a851e61-kube-api-access-dnd87\") pod \"tuned-9pvx5\" (UID: \"834e4aa4-4e23-4dc1-87a1-c8043a851e61\") " pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:14.957518 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.957499    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pz5\" (UniqueName: \"kubernetes.io/projected/130b08a5-9b01-4853-9112-f2b2847e61c3-kube-api-access-m6pz5\") pod \"multus-lhc2r\" (UID: \"130b08a5-9b01-4853-9112-f2b2847e61c3\") " pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:14.957604 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:14.957519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz7xd\" (UniqueName: \"kubernetes.io/projected/0a4bdde9-8ef0-4089-9fdf-9414452b6494-kube-api-access-sz7xd\") pod \"node-resolver-td4wf\" (UID: \"0a4bdde9-8ef0-4089-9fdf-9414452b6494\") " pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:15.029829 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.029691    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jh4g2"
Sep 16 13:08:15.037934 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.037911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:15.049197 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.049175    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m2k9b"
Sep 16 13:08:15.056162 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.056142    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rfr7l"
Sep 16 13:08:15.064221 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.064201    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lhc2r"
Sep 16 13:08:15.071357 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.071337    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9pvx5"
Sep 16 13:08:15.078318 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.078296    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:15.085558 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.085538    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-td4wf"
Sep 16 13:08:15.086670 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.086650    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:08:15.091875 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.091854    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv"
Sep 16 13:08:15.352793 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.352715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:15.352965 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.352871    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:15.352965 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.352953    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:16.352933157 +0000 UTC m=+4.029647575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:15.369222 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.369018    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2b4109_e4a9_467b_8ec7_a71ca968b738.slice/crio-bad653898e2d4a421f37e0455c5d6f62291fb90965d567a2c3f038ba77dca923 WatchSource:0}: Error finding container bad653898e2d4a421f37e0455c5d6f62291fb90965d567a2c3f038ba77dca923: Status 404 returned error can't find the container with id bad653898e2d4a421f37e0455c5d6f62291fb90965d567a2c3f038ba77dca923
Sep 16 13:08:15.374857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.373590    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce94dfbb_3a08_455c_814a_ce6009ae8682.slice/crio-fc4605f4ef2673de675294e8a707cece9f090943cb7c7a5a3b5405e82a592687 WatchSource:0}: Error finding container fc4605f4ef2673de675294e8a707cece9f090943cb7c7a5a3b5405e82a592687: Status 404 returned error can't find the container with id fc4605f4ef2673de675294e8a707cece9f090943cb7c7a5a3b5405e82a592687
Sep 16 13:08:15.374857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.374549    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d9e406_a876_4bcf_b27f_6866da5bf255.slice/crio-db42b4052f442a47d82aac29a0e53c7b340f7d7050963653869a01e1d999c710 WatchSource:0}: Error finding container db42b4052f442a47d82aac29a0e53c7b340f7d7050963653869a01e1d999c710: Status 404 returned error can't find the container with id db42b4052f442a47d82aac29a0e53c7b340f7d7050963653869a01e1d999c710
Sep 16 13:08:15.376602 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.376472    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834e4aa4_4e23_4dc1_87a1_c8043a851e61.slice/crio-784465b373fac57fdce82f531e21104149484a8fcea97084b9df9de4d909da68 WatchSource:0}: Error finding container 784465b373fac57fdce82f531e21104149484a8fcea97084b9df9de4d909da68: Status 404 returned error can't find the container with id 784465b373fac57fdce82f531e21104149484a8fcea97084b9df9de4d909da68
Sep 16 13:08:15.377017 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.376994    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53b8f25_b9fe_42b5_8069_ba6fa087d69c.slice/crio-a77875062d5391382a860200aed2f796c8aecde39ba22e4464d6aa92fa5b8c2e WatchSource:0}: Error finding container a77875062d5391382a860200aed2f796c8aecde39ba22e4464d6aa92fa5b8c2e: Status 404 returned error can't find the container with id a77875062d5391382a860200aed2f796c8aecde39ba22e4464d6aa92fa5b8c2e
Sep 16 13:08:15.401161 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.401137    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130b08a5_9b01_4853_9112_f2b2847e61c3.slice/crio-27c895f990e2646b9ea855175771b48c18b21492c55b373a4aa0e2b6e334102a WatchSource:0}: Error finding container 27c895f990e2646b9ea855175771b48c18b21492c55b373a4aa0e2b6e334102a: Status 404 returned error can't find the container with id 27c895f990e2646b9ea855175771b48c18b21492c55b373a4aa0e2b6e334102a
Sep 16 13:08:15.402860 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.402807    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3229d434_09bf_404f_b75a_bbb01395351e.slice/crio-8860bdc28ef4d4617f47c71b1b2d0de1f6c3b1c6a7f386625fffcb17dc578a77 WatchSource:0}: Error finding container 8860bdc28ef4d4617f47c71b1b2d0de1f6c3b1c6a7f386625fffcb17dc578a77: Status 404 returned error can't find the container with id 8860bdc28ef4d4617f47c71b1b2d0de1f6c3b1c6a7f386625fffcb17dc578a77
Sep 16 13:08:15.403460 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.403428    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b03a0f_8eb0_4d8f_b28e_ddaf60c2c178.slice/crio-d68619ccaf5fe57708743058a2f16f880d266b098bc58c63837af0325b293e37 WatchSource:0}: Error finding container d68619ccaf5fe57708743058a2f16f880d266b098bc58c63837af0325b293e37: Status 404 returned error can't find the container with id d68619ccaf5fe57708743058a2f16f880d266b098bc58c63837af0325b293e37
Sep 16 13:08:15.408997 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:15.408976    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4bdde9_8ef0_4089_9fdf_9414452b6494.slice/crio-dbedd7423cca0554a7eb734b80b87dd8601eff5f342d8f6e1589f46e7da9037c WatchSource:0}: Error finding container dbedd7423cca0554a7eb734b80b87dd8601eff5f342d8f6e1589f46e7da9037c: Status 404 returned error can't find the container with id dbedd7423cca0554a7eb734b80b87dd8601eff5f342d8f6e1589f46e7da9037c
Sep 16 13:08:15.453803 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.453753    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:15.453905 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.453808    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:15.453964 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.453925    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:15.454009 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.453974    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:15.454009 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.453992    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:08:16.453972664 +0000 UTC m=+4.130687081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:15.454009 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.453993    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:15.454126 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.454011    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:15.454126 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:15.454057    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:08:16.454045116 +0000 UTC m=+4.130759533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:15.779907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.779858    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:03:15 +0000 UTC" deadline="2028-06-21 19:38:22.643244494 +0000 UTC"
Sep 16 13:08:15.779907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.779905    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15462h30m6.863344049s"
Sep 16 13:08:15.802155 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.802121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-td4wf" event={"ID":"0a4bdde9-8ef0-4089-9fdf-9414452b6494","Type":"ContainerStarted","Data":"dbedd7423cca0554a7eb734b80b87dd8601eff5f342d8f6e1589f46e7da9037c"}
Sep 16 13:08:15.803480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.803456    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv" event={"ID":"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178","Type":"ContainerStarted","Data":"d68619ccaf5fe57708743058a2f16f880d266b098bc58c63837af0325b293e37"}
Sep 16 13:08:15.804946 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.804922    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nj4sb" event={"ID":"3229d434-09bf-404f-b75a-bbb01395351e","Type":"ContainerStarted","Data":"8860bdc28ef4d4617f47c71b1b2d0de1f6c3b1c6a7f386625fffcb17dc578a77"}
Sep 16 13:08:15.806406 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.806384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"db42b4052f442a47d82aac29a0e53c7b340f7d7050963653869a01e1d999c710"}
Sep 16 13:08:15.808223 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.808198    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lhc2r" event={"ID":"130b08a5-9b01-4853-9112-f2b2847e61c3","Type":"ContainerStarted","Data":"27c895f990e2646b9ea855175771b48c18b21492c55b373a4aa0e2b6e334102a"}
Sep 16 13:08:15.809521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.809497    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerStarted","Data":"a77875062d5391382a860200aed2f796c8aecde39ba22e4464d6aa92fa5b8c2e"}
Sep 16 13:08:15.810806 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.810777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9pvx5" event={"ID":"834e4aa4-4e23-4dc1-87a1-c8043a851e61","Type":"ContainerStarted","Data":"784465b373fac57fdce82f531e21104149484a8fcea97084b9df9de4d909da68"}
Sep 16 13:08:15.813987 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.813449    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal" event={"ID":"df32444e188d3dde6bc7b731c17e8041","Type":"ContainerStarted","Data":"f3a502656b591e6ecea6fcbaca43bf565e5a6c54656ed164796314b2767d91b4"}
Sep 16 13:08:15.815815 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.815790    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m2k9b" event={"ID":"ce94dfbb-3a08-455c-814a-ce6009ae8682","Type":"ContainerStarted","Data":"fc4605f4ef2673de675294e8a707cece9f090943cb7c7a5a3b5405e82a592687"}
Sep 16 13:08:15.817476 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:15.817452    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rfr7l" event={"ID":"3a2b4109-e4a9-467b-8ec7-a71ca968b738","Type":"ContainerStarted","Data":"bad653898e2d4a421f37e0455c5d6f62291fb90965d567a2c3f038ba77dca923"}
Sep 16 13:08:16.361315 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.361285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:16.361439 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.361424    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:16.361502 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.361489    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:18.361470314 +0000 UTC m=+6.038184735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.462202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.462315    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.462479    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.462496    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.462506    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.462554    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:08:18.462539584 +0000 UTC m=+6.139254012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.463281    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:16.463396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.463344    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:08:18.463326939 +0000 UTC m=+6.140041369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:16.780786 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.780688    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:03:15 +0000 UTC" deadline="2028-02-23 15:58:34.089238069 +0000 UTC"
Sep 16 13:08:16.780786 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.780729    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12602h50m17.308514013s"
Sep 16 13:08:16.797570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.796923    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:16.797570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.796944    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:16.797570 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.797061    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:16.797570 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.797394    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:16.797570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.797443    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:16.797570 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:16.797537    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:16.825909 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.825873    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal" event={"ID":"32a26a83e8371e1e188502dccce97eac","Type":"ContainerStarted","Data":"63730a6fa03327f075c8203525877558b0173cfecb83303cff1cacb98e7b3b64"}
Sep 16 13:08:16.839268 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.838631    2556 generic.go:356] "Generic (PLEG): container finished" podID="df32444e188d3dde6bc7b731c17e8041" containerID="f3a502656b591e6ecea6fcbaca43bf565e5a6c54656ed164796314b2767d91b4" exitCode=0
Sep 16 13:08:16.839268 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.838676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal" event={"ID":"df32444e188d3dde6bc7b731c17e8041","Type":"ContainerDied","Data":"f3a502656b591e6ecea6fcbaca43bf565e5a6c54656ed164796314b2767d91b4"}
Sep 16 13:08:16.845621 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:16.845095    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-82.ec2.internal" podStartSLOduration=2.845076681 podStartE2EDuration="2.845076681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:16.84389406 +0000 UTC m=+4.520608500" watchObservedRunningTime="2026-09-16 13:08:16.845076681 +0000 UTC m=+4.521791114"
Sep 16 13:08:17.853553 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:17.853445    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal" event={"ID":"df32444e188d3dde6bc7b731c17e8041","Type":"ContainerStarted","Data":"ccc787377cdf3d4e727ac89a316636d2becaedb6f80063642b7bf2e121bbfb84"}
Sep 16 13:08:18.386064 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:18.386023    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:18.386272 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.386193    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:18.386272 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.386270    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:22.386249988 +0000 UTC m=+10.062964411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:18.486854 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:18.486810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:18.487057 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:18.486866    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:18.487319 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.487154    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:18.487319 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.487177    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:18.487319 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.487191    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:18.487319 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.487254    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:08:22.487234807 +0000 UTC m=+10.163949223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:18.487755 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.487733    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:18.487840 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.487794    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:08:22.487777657 +0000 UTC m=+10.164492097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:18.797999 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:18.796994    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:18.797999 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.797137    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:18.797999 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:18.797598    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:18.797999 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.797711    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:18.797999 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:18.797775    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:18.797999 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:18.797848    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:20.793963 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:20.793932    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:20.794329 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:20.793997    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:20.794329 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:20.794114    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:20.794539 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:20.794498    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:20.794630 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:20.794549    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:20.794753 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:20.794730    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:22.423298 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:22.423264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:22.423875 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.423398    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:22.423875 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.423466    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:30.423447144 +0000 UTC m=+18.100161564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:22.524761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:22.524815    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.524975    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.524999    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.525014    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.525078    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:08:30.525056806 +0000 UTC m=+18.201771246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.525158    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:22.525327 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.525199    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:08:30.525186244 +0000 UTC m=+18.201900661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:22.795992 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:22.795917    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:22.796161 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.796044    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:22.796161 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:22.796139    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:22.796279 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.796258    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:22.798203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:22.798049    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:22.798203 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:22.798160    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:24.794664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:24.794423    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:24.795134 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:24.794421    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:24.795134 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:24.794860    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:24.795134 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:24.794773    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:24.795134 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:24.794428    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:24.795134 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:24.794958    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:26.794247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:26.794165    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:26.794247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:26.794190    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:26.794781 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:26.794179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:26.794781 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:26.794285    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:26.794781 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:26.794336    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:26.794781 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:26.794456    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:28.794031 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:28.793993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:28.794466 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:28.793993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:28.794466 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:28.794128    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:28.794466 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:28.794006    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:28.794466 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:28.794208    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:28.794466 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:28.794270    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:30.487535 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:30.487499    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:30.488033 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.487651    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:30.488033 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.487723    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:46.487705126 +0000 UTC m=+34.164419559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:30.588070 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:30.588030    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:30.588219 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:30.588072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:30.588219 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.588178    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:30.588219 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.588195    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:30.588219 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.588211    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:30.588398 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.588229    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:30.588398 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.588249    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:08:46.588228958 +0000 UTC m=+34.264943392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:30.588398 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.588276    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:08:46.588260118 +0000 UTC m=+34.264974535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:30.794762 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:30.794679    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:30.794762 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:30.794693    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:30.794762 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:30.794698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:30.795010 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.794803    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:30.795010 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.794889    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:30.795010 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:30.794940    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:32.795384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:32.795347    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:32.795888 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:32.795500    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:32.795888 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:32.795512    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:32.795888 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:32.795610    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:32.795888 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:32.795623    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:32.795888 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:32.795748    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:34.794099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.793946    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:34.794891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.793957    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:34.794891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:34.794166    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:34.794891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:34.794270    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:34.794891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.794019    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:34.794891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:34.794415    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:34.886946 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.886915    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rfr7l" event={"ID":"3a2b4109-e4a9-467b-8ec7-a71ca968b738","Type":"ContainerStarted","Data":"a6438d049eb921c7077a53cd8ba87ef4bb86afd3a38f55711b6ac7409bbdd62a"}
Sep 16 13:08:34.888754 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.888729    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-td4wf" event={"ID":"0a4bdde9-8ef0-4089-9fdf-9414452b6494","Type":"ContainerStarted","Data":"501ef635a051cc9c16dc66bbf9f123b635c6f6fc54e5fd7abfcd348819efc229"}
Sep 16 13:08:34.890655 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.890629    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv" event={"ID":"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178","Type":"ContainerStarted","Data":"3dfa2a50662880a95c961deb37c4ef17bcd471d235d324cbb14c1b8228e924dc"}
Sep 16 13:08:34.893842 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.893818    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nj4sb" event={"ID":"3229d434-09bf-404f-b75a-bbb01395351e","Type":"ContainerStarted","Data":"710f72e4f68b2ef0f4e2e829910b02ea3e3e3f396051640a5df092c87b64f1ff"}
Sep 16 13:08:34.897331 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.897311    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"5d83fb46550b5d8f9bcdf49359e1e7d7e1eb6b43eb591d7c98aa719e144449d7"}
Sep 16 13:08:34.897428 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.897377    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"72ab480246c47842614e191ae13c2f1e684e694b013cc78b40d69c81edd3ca5f"}
Sep 16 13:08:34.897489 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.897432    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"0b1d7321f21fc8f7a4fef169cc4081e3355f49b6ba896cb8e246ae2f64f7b5ad"}
Sep 16 13:08:34.897489 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.897483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"d50207a685d9c29b39fb061eabe846f72d515b84f1300323d13ac6381bb5d6eb"}
Sep 16 13:08:34.897591 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.897531    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"1d6df954610b72a45db8956146fbb28880649571f720272d3f62d6f83bd5720d"}
Sep 16 13:08:34.897591 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.897578    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"e3fb3c30e9d1d95246e37771868abb872941491e468a2bda5941e9adeb3ea914"}
Sep 16 13:08:34.899037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.899013    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lhc2r" event={"ID":"130b08a5-9b01-4853-9112-f2b2847e61c3","Type":"ContainerStarted","Data":"3cd78b0da6dd8177c547ffb15b8b5920a86bda80c908c16ef243f0b7b5591717"}
Sep 16 13:08:34.900687 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.900667    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53b8f25-b9fe-42b5-8069-ba6fa087d69c" containerID="6466882277da6d69894733bfe86fac73610ed9c4ae01a609fe4839ade470b03e" exitCode=0
Sep 16 13:08:34.900793 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.900730    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerDied","Data":"6466882277da6d69894733bfe86fac73610ed9c4ae01a609fe4839ade470b03e"}
Sep 16 13:08:34.902776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.902308    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rfr7l" podStartSLOduration=3.5867438270000003 podStartE2EDuration="21.90229084s" totalImagesPullingTime="18.315547013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:34.901695078 +0000 UTC m=+22.578409520" watchObservedRunningTime="2026-09-16 13:08:34.90229084 +0000 UTC m=+22.579005277"
Sep 16 13:08:34.902776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.902400    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9pvx5" event={"ID":"834e4aa4-4e23-4dc1-87a1-c8043a851e61","Type":"ContainerStarted","Data":"6da40c52b3326b89c96532d655c2a026e694254d7220bbd84ad043330bf4625a"}
Sep 16 13:08:34.902776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.902401    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-82.ec2.internal" podStartSLOduration=20.902392161 podStartE2EDuration="20.902392161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:17.871776232 +0000 UTC m=+5.548490671" watchObservedRunningTime="2026-09-16 13:08:34.902392161 +0000 UTC m=+22.579106615"
Sep 16 13:08:34.935208 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.935174    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nj4sb" podStartSLOduration=6.312104042 podStartE2EDuration="20.935165968s" totalImagesPullingTime="14.623061926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:34.916733324 +0000 UTC m=+22.593447759" watchObservedRunningTime="2026-09-16 13:08:34.935165968 +0000 UTC m=+22.611880385"
Sep 16 13:08:34.962523 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.962489    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-td4wf" podStartSLOduration=2.653813106 podStartE2EDuration="20.96247806s" totalImagesPullingTime="18.308664954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:34.936080358 +0000 UTC m=+22.612794794" watchObservedRunningTime="2026-09-16 13:08:34.96247806 +0000 UTC m=+22.639192549"
Sep 16 13:08:34.979257 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.979211    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lhc2r" podStartSLOduration=3.256542018 podStartE2EDuration="21.97919554s" totalImagesPullingTime="18.722653522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:34.978761622 +0000 UTC m=+22.655476058" watchObservedRunningTime="2026-09-16 13:08:34.97919554 +0000 UTC m=+22.655909978"
Sep 16 13:08:34.997389 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:34.997357    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9pvx5" podStartSLOduration=2.675389414 podStartE2EDuration="20.997345757s" totalImagesPullingTime="18.321956343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:34.99700882 +0000 UTC m=+22.673723255" watchObservedRunningTime="2026-09-16 13:08:34.997345757 +0000 UTC m=+22.674060191"
Sep 16 13:08:35.907381 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:35.907354    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m2k9b" event={"ID":"ce94dfbb-3a08-455c-814a-ce6009ae8682","Type":"ContainerStarted","Data":"201c170f871ec68b51b8a7f1ac83a7db37bf07c902d7f87607475c8a625c0f0f"}
Sep 16 13:08:35.923841 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:35.923792    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m2k9b" podStartSLOduration=4.612467341 podStartE2EDuration="22.923776782s" totalImagesPullingTime="18.311309441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:35.923434617 +0000 UTC m=+23.600149053" watchObservedRunningTime="2026-09-16 13:08:35.923776782 +0000 UTC m=+23.600491217"
Sep 16 13:08:35.954968 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:35.954946    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 13:08:36.795071 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.794921    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T13:08:35.954963222Z","UUID":"b7bf6e01-3c88-4951-8585-18413e8dcd3e","Handler":null,"Name":"","Endpoint":""}
Sep 16 13:08:36.796828 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.796804    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 13:08:36.796956 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.796864    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 13:08:36.798657 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.798636    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:36.798776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.798672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:36.798776 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:36.798745    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:36.798776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.798643    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:36.798935 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:36.798858    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:36.798989 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:36.798973    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:36.911494 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.911459    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv" event={"ID":"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178","Type":"ContainerStarted","Data":"4d5817ad9688aeab6fea5e2744a2e71c3e71c76e54b50b108e033fd17154192b"}
Sep 16 13:08:36.915721 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:36.915691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"40440ecc806ba6fecbb904a43f3b4afbda65efd6bf037480365453f5eaebc5e6"}
Sep 16 13:08:37.920902 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:37.920853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv" event={"ID":"e6b03a0f-8eb0-4d8f-b28e-ddaf60c2c178","Type":"ContainerStarted","Data":"fa3f6599f453e2f781aa9e308c61c431bfc9f13d355fc48c5720aa883b8a61be"}
Sep 16 13:08:37.941416 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:37.941371    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnntv" podStartSLOduration=2.728748989 podStartE2EDuration="23.941358106s" totalImagesPullingTime="21.212609117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:37.941111005 +0000 UTC m=+25.617825453" watchObservedRunningTime="2026-09-16 13:08:37.941358106 +0000 UTC m=+25.618072540"
Sep 16 13:08:38.798056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:38.798031    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:38.798056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:38.798067    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:38.798433 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:38.798163    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:38.798433 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:38.798164    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:38.798433 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:38.798246    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:38.798433 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:38.798314    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:38.927812 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:38.927728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" event={"ID":"72d9e406-a876-4bcf-b27f-6866da5bf255","Type":"ContainerStarted","Data":"11df53ae071c8ff0460932aee731057695c078e77894307803170bbaa25b9acd"}
Sep 16 13:08:39.797085 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.797049    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:39.797808 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.797784    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:39.811972 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.811931    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" podStartSLOduration=8.453647348 podStartE2EDuration="26.811916736s" totalImagesPullingTime="18.358269388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:38.952957146 +0000 UTC m=+26.629671591" watchObservedRunningTime="2026-09-16 13:08:39.811916736 +0000 UTC m=+27.488631241"
Sep 16 13:08:39.931457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.930756    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 13:08:39.931931 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.931768    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:39.931931 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.931837    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:39.931931 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.931913    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:39.932495 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.932460    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nj4sb"
Sep 16 13:08:39.963703 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.963671    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:39.968616 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:39.965923    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:40.575660 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.575571    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jtk45"]
Sep 16 13:08:40.575769 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.575735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:40.575862 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:40.575842    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:40.579263 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.579237    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2n6bp"]
Sep 16 13:08:40.579375 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.579341    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:40.579473 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:40.579427    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:40.580030 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.580000    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jdkss"]
Sep 16 13:08:40.580150 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.580135    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:40.580269 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:40.580249    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:40.934266 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.934238    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53b8f25-b9fe-42b5-8069-ba6fa087d69c" containerID="b2e54c7a93efb3c39a22b6b180b765f7182c5466960370b36567c6db4261b0b8" exitCode=0
Sep 16 13:08:40.934765 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.934327    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerDied","Data":"b2e54c7a93efb3c39a22b6b180b765f7182c5466960370b36567c6db4261b0b8"}
Sep 16 13:08:40.934765 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:40.934531    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 13:08:41.794049 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:41.794024    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:41.794198 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:41.794122    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:41.936957 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:41.936926    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 13:08:42.795388 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:42.795364    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:42.795504 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:42.795438    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:42.795541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:42.795523    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:42.795640 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:42.795619    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:42.940252 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:42.940222    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53b8f25-b9fe-42b5-8069-ba6fa087d69c" containerID="f10f1df3d7026acd1f13390641f7a37a8122861860a723efb08e014d9320ee12" exitCode=0
Sep 16 13:08:42.940580 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:42.940264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerDied","Data":"f10f1df3d7026acd1f13390641f7a37a8122861860a723efb08e014d9320ee12"}
Sep 16 13:08:43.794649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:43.794427    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:43.794789 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:43.794767    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:44.404010 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.403981    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:08:44.404477 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.404176    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 13:08:44.416004 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.415939    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" podUID="72d9e406-a876-4bcf-b27f-6866da5bf255" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 16 13:08:44.423961 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.423936    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv" podUID="72d9e406-a876-4bcf-b27f-6866da5bf255" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 16 13:08:44.794316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.794245    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:44.794537 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:44.794340    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6bp" podUID="694264fb-023d-4197-b834-514ec068625c"
Sep 16 13:08:44.794537 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.794414    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:44.794537 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:44.794490    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtk45" podUID="94ffed1b-cdef-48f6-90c6-a3ce87d96c4b"
Sep 16 13:08:44.946998 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.946971    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53b8f25-b9fe-42b5-8069-ba6fa087d69c" containerID="e80c8c39c7f26a492e705ecc5dc19186905aea2d07a43c32a4f1cf8331221e91" exitCode=0
Sep 16 13:08:44.947203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:44.947011    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerDied","Data":"e80c8c39c7f26a492e705ecc5dc19186905aea2d07a43c32a4f1cf8331221e91"}
Sep 16 13:08:45.794843 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:45.794765    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:45.795245 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:45.794942    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdkss" podUID="c4c5e114-51f5-4670-85af-f57c14bc6c58"
Sep 16 13:08:46.514852 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.514815    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:46.514979 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.514950    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:46.515020 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.515011    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:18.514997608 +0000 UTC m=+66.191712020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:08:46.615884 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.615854    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:46.615884 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.615885    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:46.616046 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.615996    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:46.616046 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.616011    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:08:46.616046 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.616023    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:08:46.616046 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.616034    2556 projected.go:196] Error preparing data for projected volume kube-api-access-8jzvc for pod openshift-network-diagnostics/network-check-target-jtk45: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:46.616190 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.616050    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret podName:694264fb-023d-4197-b834-514ec068625c nodeName:}" failed. No retries permitted until 2026-09-16 13:09:18.616033061 +0000 UTC m=+66.292747491 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret") pod "global-pull-secret-syncer-2n6bp" (UID: "694264fb-023d-4197-b834-514ec068625c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:08:46.616190 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.616068    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc podName:94ffed1b-cdef-48f6-90c6-a3ce87d96c4b nodeName:}" failed. No retries permitted until 2026-09-16 13:09:18.616058429 +0000 UTC m=+66.292772842 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8jzvc" (UniqueName: "kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc") pod "network-check-target-jtk45" (UID: "94ffed1b-cdef-48f6-90c6-a3ce87d96c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:08:46.656562 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.656541    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-82.ec2.internal" event="NodeReady"
Sep 16 13:08:46.656691 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.656680    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 13:08:46.689729 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.689702    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dc5678fd9-5qr4n"]
Sep 16 13:08:46.692229 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.692212    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.694388 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.694364    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 13:08:46.694497 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.694471    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 13:08:46.694679 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.694661    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 13:08:46.694764 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.694689    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lhrr5\""
Sep 16 13:08:46.701290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.700988    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 13:08:46.705105 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.705089    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hn46r"]
Sep 16 13:08:46.707079 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.707055    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.708259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.708240    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dc5678fd9-5qr4n"]
Sep 16 13:08:46.709520 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.709389    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 13:08:46.709520 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.709404    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 13:08:46.709520 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.709412    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ftxst\""
Sep 16 13:08:46.711310 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.711290    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dlp96"]
Sep 16 13:08:46.713242 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.713223    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.715708 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.715688    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dbtv4\""
Sep 16 13:08:46.715813 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.715777    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 13:08:46.715813 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.715804    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 13:08:46.716110 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.716091    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 13:08:46.724538 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.724521    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dlp96"]
Sep 16 13:08:46.725743 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.725713    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hn46r"]
Sep 16 13:08:46.794063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.794013    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:08:46.794189 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.794173    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:08:46.796377 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.796360    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 13:08:46.796735 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.796383    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 13:08:46.796735 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.796392    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 13:08:46.796735 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.796501    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sfm4x\""
Sep 16 13:08:46.816868 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.816851    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pgz8\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-kube-api-access-8pgz8\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.816967 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.816887    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-image-registry-private-configuration\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.816967 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.816907    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24e6abeb-d83c-4f33-821c-e0348b063346-ca-trust-extracted\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.816967 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.816923    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6c759f5e-28c4-4176-9f32-b75b43efc524-config-volume\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.816967 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.816936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c759f5e-28c4-4176-9f32-b75b43efc524-tmp-dir\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.817178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817000    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-installation-pull-secrets\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.817178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.817178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817069    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngbdr\" (UniqueName: \"kubernetes.io/projected/6c759f5e-28c4-4176-9f32-b75b43efc524-kube-api-access-ngbdr\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.817178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817096    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lscs\" (UniqueName: \"kubernetes.io/projected/8455cf6b-b3c9-4c05-b62d-613ed31ba4f0-kube-api-access-2lscs\") pod \"ingress-canary-dlp96\" (UID: \"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0\") " pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.817178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.817383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-registry-certificates\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.817383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817228    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8455cf6b-b3c9-4c05-b62d-613ed31ba4f0-cert\") pod \"ingress-canary-dlp96\" (UID: \"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0\") " pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.817383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817255    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-trusted-ca\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.817383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.817285    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-bound-sa-token\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.917650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.917749 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917657    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngbdr\" (UniqueName: \"kubernetes.io/projected/6c759f5e-28c4-4176-9f32-b75b43efc524-kube-api-access-ngbdr\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.917749 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917684    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lscs\" (UniqueName: \"kubernetes.io/projected/8455cf6b-b3c9-4c05-b62d-613ed31ba4f0-kube-api-access-2lscs\") pod \"ingress-canary-dlp96\" (UID: \"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0\") " pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.917749 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917737    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.917760    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917780    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-registry-certificates\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.917809    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:47.417795689 +0000 UTC m=+35.094510102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917830    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8455cf6b-b3c9-4c05-b62d-613ed31ba4f0-cert\") pod \"ingress-canary-dlp96\" (UID: \"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0\") " pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917851    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-trusted-ca\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.917858    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917867    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-bound-sa-token\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.917874    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:08:46.917914 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:46.917920    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:47.417904417 +0000 UTC m=+35.094618830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:08:46.918341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pgz8\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-kube-api-access-8pgz8\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.918341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.917979    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-image-registry-private-configuration\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.918341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918013    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24e6abeb-d83c-4f33-821c-e0348b063346-ca-trust-extracted\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.918341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918040    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6c759f5e-28c4-4176-9f32-b75b43efc524-config-volume\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.918341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918062    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c759f5e-28c4-4176-9f32-b75b43efc524-tmp-dir\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.918341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918091    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-installation-pull-secrets\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.918664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918403    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24e6abeb-d83c-4f33-821c-e0348b063346-ca-trust-extracted\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.918664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918407    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-registry-certificates\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.918664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918655    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c759f5e-28c4-4176-9f32-b75b43efc524-tmp-dir\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.918781 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918699    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6c759f5e-28c4-4176-9f32-b75b43efc524-config-volume\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.918781 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.918765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-trusted-ca\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.921780 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.921754    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-image-registry-private-configuration\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.921864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.921805    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-installation-pull-secrets\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.921976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.921956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8455cf6b-b3c9-4c05-b62d-613ed31ba4f0-cert\") pod \"ingress-canary-dlp96\" (UID: \"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0\") " pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.926592 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.926565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lscs\" (UniqueName: \"kubernetes.io/projected/8455cf6b-b3c9-4c05-b62d-613ed31ba4f0-kube-api-access-2lscs\") pod \"ingress-canary-dlp96\" (UID: \"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0\") " pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:46.927095 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.927074    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngbdr\" (UniqueName: \"kubernetes.io/projected/6c759f5e-28c4-4176-9f32-b75b43efc524-kube-api-access-ngbdr\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:46.927196 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.927155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pgz8\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-kube-api-access-8pgz8\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:46.927453 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:46.927436    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-bound-sa-token\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:47.025275 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.025250    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dlp96"
Sep 16 13:08:47.305960 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.305726    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dlp96"]
Sep 16 13:08:47.311831 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:08:47.311802    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8455cf6b_b3c9_4c05_b62d_613ed31ba4f0.slice/crio-a0e1f26fff96f22d22c31d6c62b8ee99944a398d25f6df446310dc9c475c80a3 WatchSource:0}: Error finding container a0e1f26fff96f22d22c31d6c62b8ee99944a398d25f6df446310dc9c475c80a3: Status 404 returned error can't find the container with id a0e1f26fff96f22d22c31d6c62b8ee99944a398d25f6df446310dc9c475c80a3
Sep 16 13:08:47.422518 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.422492    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:47.422749 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.422565    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:47.422749 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:47.422659    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:08:47.422749 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:47.422680    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:08:47.422749 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:47.422723    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:48.422709129 +0000 UTC m=+36.099423546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:08:47.422749 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:47.422662    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:08:47.423054 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:47.422780    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:48.422768928 +0000 UTC m=+36.099483347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:08:47.794514 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.794450    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:08:47.796791 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.796744    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 13:08:47.797104 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.796944    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f5rcg\""
Sep 16 13:08:47.955772 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:47.955742    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dlp96" event={"ID":"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0","Type":"ContainerStarted","Data":"a0e1f26fff96f22d22c31d6c62b8ee99944a398d25f6df446310dc9c475c80a3"}
Sep 16 13:08:48.430650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:48.430610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:48.430828 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:48.430683    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:48.430828 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:48.430708    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:08:48.430828 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:48.430771    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:50.430753109 +0000 UTC m=+38.107467539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:08:48.430828 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:48.430814    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:08:48.430828 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:48.430825    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:08:48.431072 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:48.430858    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:50.430847527 +0000 UTC m=+38.107561961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:08:50.443147 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:50.443068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:50.443147 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:50.443128    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:50.443617 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:50.443230    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:08:50.443617 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:50.443246    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:08:50.443617 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:50.443231    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:08:50.443617 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:50.443305    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:54.443292093 +0000 UTC m=+42.120006525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:08:50.443617 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:50.443348    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:08:54.443328099 +0000 UTC m=+42.120042515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:08:50.518773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:50.518731    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:50.606384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:50.606344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:50.818112 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:50.818063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:51.010850 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:51.010780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:51.424097 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:51.424063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:51.527245 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:51.527174    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:51.968218 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:51.968192    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53b8f25-b9fe-42b5-8069-ba6fa087d69c" containerID="19c55bcb82d76d559bf6af14d6b43c2528026f587eadd0f3a979a2e117fe0929" exitCode=0
Sep 16 13:08:51.968365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:51.968283    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerDied","Data":"19c55bcb82d76d559bf6af14d6b43c2528026f587eadd0f3a979a2e117fe0929"}
Sep 16 13:08:51.969779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:51.969755    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dlp96" event={"ID":"8455cf6b-b3c9-4c05-b62d-613ed31ba4f0","Type":"ContainerStarted","Data":"5e004c508768d7fc038b1d2610bbf5a2438ac2eb735dec8bdcc9cc8e6dc43b2c"}
Sep 16 13:08:52.004162 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.004114    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dlp96" podStartSLOduration=1.8376428439999999 podStartE2EDuration="6.004095362s" totalImagesPullingTime="4.166452518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:52.003226114 +0000 UTC m=+39.679940552" watchObservedRunningTime="2026-09-16 13:08:52.004095362 +0000 UTC m=+39.680809811"
Sep 16 13:08:52.203726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.203701    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:52.304366 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.304284    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:52.640492 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.640456    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:52.741466 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.741409    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:52.973892 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.973840    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53b8f25-b9fe-42b5-8069-ba6fa087d69c" containerID="bbee071736bf30137e8c2de34b9ffd3bc60268b41f859760637d6925e015fdab" exitCode=0
Sep 16 13:08:52.973993 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:52.973926    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerDied","Data":"bbee071736bf30137e8c2de34b9ffd3bc60268b41f859760637d6925e015fdab"}
Sep 16 13:08:53.980183 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:53.980146    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" event={"ID":"e53b8f25-b9fe-42b5-8069-ba6fa087d69c","Type":"ContainerStarted","Data":"b92c3975b6c0bc664fbf59dd2de6f76392923a525f3a7be9d83b015c4f0bdd01"}
Sep 16 13:08:54.000977 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:54.000937    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jh4g2" podStartSLOduration=7.85504291 podStartE2EDuration="41.000923029s" totalImagesPullingTime="33.145880119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:08:54.000321066 +0000 UTC m=+41.677035525" watchObservedRunningTime="2026-09-16 13:08:54.000923029 +0000 UTC m=+41.677637461"
Sep 16 13:08:54.121642 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:54.121612    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:54.222470 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:54.222437    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:54.471470 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:54.471438    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:08:54.471635 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:54.471502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:08:54.471635 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:54.471586    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:08:54.471635 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:54.471626    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:08:54.471757 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:54.471675    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:02.471661202 +0000 UTC m=+50.148375621 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:08:54.471757 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:54.471591    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:08:54.471757 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:08:54.471730    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:02.471719172 +0000 UTC m=+50.148433586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:08:56.646872 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:56.646831    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:08:56.747130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:08:56.747097    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log" limit=1024
Sep 16 13:09:02.527339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:02.527289    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:09:02.527904 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:02.527361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:09:02.527904 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:02.527459    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:09:02.527904 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:02.527513    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:09:02.527904 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:02.527535    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:09:02.527904 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:02.527545    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:18.527529141 +0000 UTC m=+66.204243554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:09:02.527904 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:02.527593    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:18.527579695 +0000 UTC m=+66.204294109 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:09:03.648646 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.648608    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"]
Sep 16 13:09:03.663515 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.663493    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.663515 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.663504    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"]
Sep 16 13:09:03.666956 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.666936    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 13:09:03.667082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.666982    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 13:09:03.667082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.667029    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 16 13:09:03.667082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.666939    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 16 13:09:03.667082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.666939    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-446mb\""
Sep 16 13:09:03.735086 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.735059    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.735192 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.735118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e1502655-4118-4947-86c7-0040ae5a3791-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.735192 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.735175    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-246lw\" (UniqueName: \"kubernetes.io/projected/e1502655-4118-4947-86c7-0040ae5a3791-kube-api-access-246lw\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.746703 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.746677    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-675764b9db-82s2n"]
Sep 16 13:09:03.766532 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.766508    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-675764b9db-82s2n"]
Sep 16 13:09:03.766633 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.766575    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.771061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.771043    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 13:09:03.771061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.771054    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 13:09:03.772244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.772228    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 13:09:03.772556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.772460    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 13:09:03.772982 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.772965    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 13:09:03.773203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.773191    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6fr7k\""
Sep 16 13:09:03.775074 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.775060    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 13:09:03.836418 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.836487 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836420    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.836487 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836456    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.836564 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:03.836524    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:03.836564 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836546    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e1502655-4118-4947-86c7-0040ae5a3791-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.836649 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:03.836572    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls podName:e1502655-4118-4947-86c7-0040ae5a3791 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:04.336558186 +0000 UTC m=+52.013272623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7bv2q" (UID: "e1502655-4118-4947-86c7-0040ae5a3791") : secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:03.836649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836589    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-default-certificate\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.836649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836624    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz9nj\" (UniqueName: \"kubernetes.io/projected/01e0e7b1-ef76-4103-99cf-588da4736933-kube-api-access-tz9nj\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.836752 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836652    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-246lw\" (UniqueName: \"kubernetes.io/projected/e1502655-4118-4947-86c7-0040ae5a3791-kube-api-access-246lw\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.836752 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.836717    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-stats-auth\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.837218 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.837199    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e1502655-4118-4947-86c7-0040ae5a3791-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.850164 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.850141    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"]
Sep 16 13:09:03.857477 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.857457    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-246lw\" (UniqueName: \"kubernetes.io/projected/e1502655-4118-4947-86c7-0040ae5a3791-kube-api-access-246lw\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:03.864017 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.863998    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"
Sep 16 13:09:03.864760 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.864741    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"]
Sep 16 13:09:03.866392 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.866373    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 13:09:03.866500 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.866434    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-l99fd\""
Sep 16 13:09:03.866562 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.866523    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:09:03.937335 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.937307    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-stats-auth\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.937442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.937348    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qlfn\" (UniqueName: \"kubernetes.io/projected/f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd-kube-api-access-7qlfn\") pod \"volume-data-source-validator-86695b7cc7-jczcv\" (UID: \"f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"
Sep 16 13:09:03.937442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.937389    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.937442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.937415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.937625 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.937523    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-default-certificate\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.937625 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.937547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz9nj\" (UniqueName: \"kubernetes.io/projected/01e0e7b1-ef76-4103-99cf-588da4736933-kube-api-access-tz9nj\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.937625 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:03.937524    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 13:09:03.937794 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:03.937560    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:04.437543294 +0000 UTC m=+52.114257706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : configmap references non-existent config key: service-ca.crt
Sep 16 13:09:03.937794 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:03.937691    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:04.43767304 +0000 UTC m=+52.114387462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : secret "router-metrics-certs-default" not found
Sep 16 13:09:03.939639 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.939614    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-default-certificate\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.939717 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.939653    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-stats-auth\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.949799 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.949780    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz9nj\" (UniqueName: \"kubernetes.io/projected/01e0e7b1-ef76-4103-99cf-588da4736933-kube-api-access-tz9nj\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:03.951537 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.951517    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-v86g5"]
Sep 16 13:09:03.966931 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.966916    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:03.966931 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.966923    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-878hd"]
Sep 16 13:09:03.969424 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.969405    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 13:09:03.969536 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.969449    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 13:09:03.969536 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.969411    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 13:09:03.969536 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.969508    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 13:09:03.969706 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.969620    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8jtfm\""
Sep 16 13:09:03.979390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.979363    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 13:09:03.980507 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.980466    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"]
Sep 16 13:09:03.980623 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.980587    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:03.982689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.982672    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 13:09:03.983209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.983192    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 13:09:03.983301 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.983242    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 13:09:03.983301 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.983257    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:09:03.983434 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.983326    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6627k\""
Sep 16 13:09:03.988883 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.988865    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 13:09:03.998916 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.998897    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-v86g5"]
Sep 16 13:09:03.999002 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.998929    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-878hd"]
Sep 16 13:09:03.999002 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.998951    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"]
Sep 16 13:09:03.999002 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:03.998982    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.001271 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.001246    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 13:09:04.001366 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.001246    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 13:09:04.001366 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.001320    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lzq8b\""
Sep 16 13:09:04.001709 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.001695    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:09:04.038279 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038255    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.038372 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038293    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1219ffb1-73d9-4ea2-a992-2d379213dc9f-snapshots\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.038372 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038316    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91d784c7-fff8-4c82-9f66-70abca2656e9-trusted-ca\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.038372 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038342    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1219ffb1-73d9-4ea2-a992-2d379213dc9f-service-ca-bundle\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.038505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qlfn\" (UniqueName: \"kubernetes.io/projected/f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd-kube-api-access-7qlfn\") pod \"volume-data-source-validator-86695b7cc7-jczcv\" (UID: \"f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"
Sep 16 13:09:04.038505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ww6l\" (UniqueName: \"kubernetes.io/projected/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-kube-api-access-6ww6l\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.038505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038414    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91d784c7-fff8-4c82-9f66-70abca2656e9-serving-cert\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.038505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038439    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1219ffb1-73d9-4ea2-a992-2d379213dc9f-serving-cert\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.038505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038453    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrmp7\" (UniqueName: \"kubernetes.io/projected/1219ffb1-73d9-4ea2-a992-2d379213dc9f-kube-api-access-nrmp7\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.038505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038468    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91d784c7-fff8-4c82-9f66-70abca2656e9-config\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.038726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038488    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1219ffb1-73d9-4ea2-a992-2d379213dc9f-trusted-ca-bundle\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.038726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038588    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rrcf\" (UniqueName: \"kubernetes.io/projected/91d784c7-fff8-4c82-9f66-70abca2656e9-kube-api-access-4rrcf\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.038726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.038645    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1219ffb1-73d9-4ea2-a992-2d379213dc9f-tmp\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.046433 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.046413    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qlfn\" (UniqueName: \"kubernetes.io/projected/f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd-kube-api-access-7qlfn\") pod \"volume-data-source-validator-86695b7cc7-jczcv\" (UID: \"f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"
Sep 16 13:09:04.139175 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1219ffb1-73d9-4ea2-a992-2d379213dc9f-service-ca-bundle\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ww6l\" (UniqueName: \"kubernetes.io/projected/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-kube-api-access-6ww6l\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.139253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139207    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91d784c7-fff8-4c82-9f66-70abca2656e9-serving-cert\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.139381 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1219ffb1-73d9-4ea2-a992-2d379213dc9f-serving-cert\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139449 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139397    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrmp7\" (UniqueName: \"kubernetes.io/projected/1219ffb1-73d9-4ea2-a992-2d379213dc9f-kube-api-access-nrmp7\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139449 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139424    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91d784c7-fff8-4c82-9f66-70abca2656e9-config\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.139556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139449    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1219ffb1-73d9-4ea2-a992-2d379213dc9f-trusted-ca-bundle\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139556 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139489    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rrcf\" (UniqueName: \"kubernetes.io/projected/91d784c7-fff8-4c82-9f66-70abca2656e9-kube-api-access-4rrcf\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.139731 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139708    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1219ffb1-73d9-4ea2-a992-2d379213dc9f-tmp\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139749    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.139800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139753    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1219ffb1-73d9-4ea2-a992-2d379213dc9f-service-ca-bundle\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139954 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.139821    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 13:09:04.139954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139864    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1219ffb1-73d9-4ea2-a992-2d379213dc9f-snapshots\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.139954 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.139877    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls podName:9daa4d5e-c75c-4953-bdbd-f288ecb9b654 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:04.63985559 +0000 UTC m=+52.316570008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-mzx6j" (UID: "9daa4d5e-c75c-4953-bdbd-f288ecb9b654") : secret "samples-operator-tls" not found
Sep 16 13:09:04.139954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.139902    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91d784c7-fff8-4c82-9f66-70abca2656e9-trusted-ca\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.140157 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.140067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1219ffb1-73d9-4ea2-a992-2d379213dc9f-tmp\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.140326 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.140306    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1219ffb1-73d9-4ea2-a992-2d379213dc9f-trusted-ca-bundle\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.140505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.140487    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1219ffb1-73d9-4ea2-a992-2d379213dc9f-snapshots\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.141517 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.141502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1219ffb1-73d9-4ea2-a992-2d379213dc9f-serving-cert\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.144875 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.144856    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91d784c7-fff8-4c82-9f66-70abca2656e9-config\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.144984 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.144966    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91d784c7-fff8-4c82-9f66-70abca2656e9-trusted-ca\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.146216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.146197    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91d784c7-fff8-4c82-9f66-70abca2656e9-serving-cert\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.148959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.148931    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rrcf\" (UniqueName: \"kubernetes.io/projected/91d784c7-fff8-4c82-9f66-70abca2656e9-kube-api-access-4rrcf\") pod \"console-operator-696884fb9c-878hd\" (UID: \"91d784c7-fff8-4c82-9f66-70abca2656e9\") " pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.149971 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.149950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrmp7\" (UniqueName: \"kubernetes.io/projected/1219ffb1-73d9-4ea2-a992-2d379213dc9f-kube-api-access-nrmp7\") pod \"insights-operator-5844b856db-v86g5\" (UID: \"1219ffb1-73d9-4ea2-a992-2d379213dc9f\") " pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.150336 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.150318    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ww6l\" (UniqueName: \"kubernetes.io/projected/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-kube-api-access-6ww6l\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.172896 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.172851    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"
Sep 16 13:09:04.278877 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.278849    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-v86g5"
Sep 16 13:09:04.289002 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.288977    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:04.341989 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.341771    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:04.342118 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.342052    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:04.342179 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.342124    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls podName:e1502655-4118-4947-86c7-0040ae5a3791 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:05.342105698 +0000 UTC m=+53.018820126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7bv2q" (UID: "e1502655-4118-4947-86c7-0040ae5a3791") : secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:04.442528 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.442468    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:04.442671 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.442551    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:04.442727 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.442669    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:05.44265133 +0000 UTC m=+53.119365762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : configmap references non-existent config key: service-ca.crt
Sep 16 13:09:04.442727 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.442702    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 13:09:04.442805 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.442758    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:05.44274414 +0000 UTC m=+53.119458553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : secret "router-metrics-certs-default" not found
Sep 16 13:09:04.459764 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.459740    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv"]
Sep 16 13:09:04.462664 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:04.462642    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1dcc4c6_5d6e_42d9_8eca_7ffc980e19cd.slice/crio-b738f8823ce8cc73b11e34dde719c222a91bdfc5617054671e92b7200cd02a12 WatchSource:0}: Error finding container b738f8823ce8cc73b11e34dde719c222a91bdfc5617054671e92b7200cd02a12: Status 404 returned error can't find the container with id b738f8823ce8cc73b11e34dde719c222a91bdfc5617054671e92b7200cd02a12
Sep 16 13:09:04.557560 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.557534    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-v86g5"]
Sep 16 13:09:04.561145 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:04.561121    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1219ffb1_73d9_4ea2_a992_2d379213dc9f.slice/crio-10d9e899b8a96c4c7f8b9e9cf4001317ce68b71819609fabc4c2fd9019acb1eb WatchSource:0}: Error finding container 10d9e899b8a96c4c7f8b9e9cf4001317ce68b71819609fabc4c2fd9019acb1eb: Status 404 returned error can't find the container with id 10d9e899b8a96c4c7f8b9e9cf4001317ce68b71819609fabc4c2fd9019acb1eb
Sep 16 13:09:04.575089 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.575067    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-878hd"]
Sep 16 13:09:04.643854 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:04.643828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:04.644005 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.643985    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 13:09:04.644073 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:04.644057    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls podName:9daa4d5e-c75c-4953-bdbd-f288ecb9b654 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:05.644036718 +0000 UTC m=+53.320751134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-mzx6j" (UID: "9daa4d5e-c75c-4953-bdbd-f288ecb9b654") : secret "samples-operator-tls" not found
Sep 16 13:09:05.006084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.006044    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-878hd" event={"ID":"91d784c7-fff8-4c82-9f66-70abca2656e9","Type":"ContainerStarted","Data":"7761d9826facad65ea3a41820df719a8643959e5356f5bb6a38a1617c57517b3"}
Sep 16 13:09:05.007096 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.007041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-v86g5" event={"ID":"1219ffb1-73d9-4ea2-a992-2d379213dc9f","Type":"ContainerStarted","Data":"10d9e899b8a96c4c7f8b9e9cf4001317ce68b71819609fabc4c2fd9019acb1eb"}
Sep 16 13:09:05.008104 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.008083    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv" event={"ID":"f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd","Type":"ContainerStarted","Data":"b738f8823ce8cc73b11e34dde719c222a91bdfc5617054671e92b7200cd02a12"}
Sep 16 13:09:05.350644 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.350364    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:05.350644 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.350497    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:05.350644 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.350568    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls podName:e1502655-4118-4947-86c7-0040ae5a3791 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:07.350548437 +0000 UTC m=+55.027262853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7bv2q" (UID: "e1502655-4118-4947-86c7-0040ae5a3791") : secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:05.451714 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.451681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:05.451855 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.451794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:05.451925 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.451886    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:07.451864356 +0000 UTC m=+55.128578781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : configmap references non-existent config key: service-ca.crt
Sep 16 13:09:05.452003 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.451979    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 13:09:05.452123 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.452040    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:07.452023635 +0000 UTC m=+55.128738060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : secret "router-metrics-certs-default" not found
Sep 16 13:09:05.653247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:05.653204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:05.653450 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.653424    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 13:09:05.653554 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:05.653504    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls podName:9daa4d5e-c75c-4953-bdbd-f288ecb9b654 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:07.653483175 +0000 UTC m=+55.330197614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-mzx6j" (UID: "9daa4d5e-c75c-4953-bdbd-f288ecb9b654") : secret "samples-operator-tls" not found
Sep 16 13:09:06.742175 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.742146    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"]
Sep 16 13:09:06.744097 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.744074    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.746130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.746105    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 16 13:09:06.746251 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.746212    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 16 13:09:06.746907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.746888    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5nhxn\""
Sep 16 13:09:06.747014 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.746925    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 16 13:09:06.747081 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.747011    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:09:06.752842 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.752429    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"]
Sep 16 13:09:06.762179 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.762156    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d420b4ec-b559-4e74-964a-1bacec5229e5-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.762267 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.762218    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbv2g\" (UniqueName: \"kubernetes.io/projected/d420b4ec-b559-4e74-964a-1bacec5229e5-kube-api-access-jbv2g\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.762338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.762303    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d420b4ec-b559-4e74-964a-1bacec5229e5-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.863538 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.863508    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d420b4ec-b559-4e74-964a-1bacec5229e5-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.863693 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.863569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv2g\" (UniqueName: \"kubernetes.io/projected/d420b4ec-b559-4e74-964a-1bacec5229e5-kube-api-access-jbv2g\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.863693 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.863645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d420b4ec-b559-4e74-964a-1bacec5229e5-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.864140 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.864119    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d420b4ec-b559-4e74-964a-1bacec5229e5-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.866159 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.866136    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d420b4ec-b559-4e74-964a-1bacec5229e5-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:06.871892 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:06.871868    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbv2g\" (UniqueName: \"kubernetes.io/projected/d420b4ec-b559-4e74-964a-1bacec5229e5-kube-api-access-jbv2g\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gjlt\" (UID: \"d420b4ec-b559-4e74-964a-1bacec5229e5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:07.054332 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:07.054255    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"
Sep 16 13:09:07.367647 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:07.367614    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:07.367777 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.367700    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:07.367777 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.367760    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls podName:e1502655-4118-4947-86c7-0040ae5a3791 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:11.367745382 +0000 UTC m=+59.044459794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7bv2q" (UID: "e1502655-4118-4947-86c7-0040ae5a3791") : secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:07.468281 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:07.468247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:07.468404 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:07.468319    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:07.468453 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.468420    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:11.468402409 +0000 UTC m=+59.145116842 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : configmap references non-existent config key: service-ca.crt
Sep 16 13:09:07.468503 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.468475    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 13:09:07.468546 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.468528    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:11.468515455 +0000 UTC m=+59.145229867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : secret "router-metrics-certs-default" not found
Sep 16 13:09:07.504540 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:07.504515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt"]
Sep 16 13:09:07.507863 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:07.507839    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd420b4ec_b559_4e74_964a_1bacec5229e5.slice/crio-26db32da6709f3e6d8c87cb6e7e800908a39179b83df300c0f4f49c80e2ef3de WatchSource:0}: Error finding container 26db32da6709f3e6d8c87cb6e7e800908a39179b83df300c0f4f49c80e2ef3de: Status 404 returned error can't find the container with id 26db32da6709f3e6d8c87cb6e7e800908a39179b83df300c0f4f49c80e2ef3de
Sep 16 13:09:07.669409 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:07.669387    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:07.669552 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.669533    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 13:09:07.669642 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:07.669618    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls podName:9daa4d5e-c75c-4953-bdbd-f288ecb9b654 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:11.669583427 +0000 UTC m=+59.346297864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-mzx6j" (UID: "9daa4d5e-c75c-4953-bdbd-f288ecb9b654") : secret "samples-operator-tls" not found
Sep 16 13:09:08.016334 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.016275    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv" event={"ID":"f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd","Type":"ContainerStarted","Data":"6e06cdb31210c22796b13a16b539cd462517d22210c90ddf63c357de1f8d2416"}
Sep 16 13:09:08.017894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.017877    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/0.log"
Sep 16 13:09:08.017973 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.017909    2556 generic.go:356] "Generic (PLEG): container finished" podID="91d784c7-fff8-4c82-9f66-70abca2656e9" containerID="1eefc87b9895abc99ed0b85c39e50f95bec06b053e20767747845e00b15b55be" exitCode=255
Sep 16 13:09:08.018011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.017990    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-878hd" event={"ID":"91d784c7-fff8-4c82-9f66-70abca2656e9","Type":"ContainerDied","Data":"1eefc87b9895abc99ed0b85c39e50f95bec06b053e20767747845e00b15b55be"}
Sep 16 13:09:08.018144 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.018130    2556 scope.go:117] "RemoveContainer" containerID="1eefc87b9895abc99ed0b85c39e50f95bec06b053e20767747845e00b15b55be"
Sep 16 13:09:08.019649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.019630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-v86g5" event={"ID":"1219ffb1-73d9-4ea2-a992-2d379213dc9f","Type":"ContainerStarted","Data":"735729bdae33c20ec56a6f6edb8e4132e002013daf20afe987a4aad747d594d3"}
Sep 16 13:09:08.020801 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.020782    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt" event={"ID":"d420b4ec-b559-4e74-964a-1bacec5229e5","Type":"ContainerStarted","Data":"26db32da6709f3e6d8c87cb6e7e800908a39179b83df300c0f4f49c80e2ef3de"}
Sep 16 13:09:08.031909 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.031871    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jczcv" podStartSLOduration=2.262170807 podStartE2EDuration="5.03185844s" totalImagesPullingTime="2.769687633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:08.031218387 +0000 UTC m=+55.707932849" watchObservedRunningTime="2026-09-16 13:09:08.03185844 +0000 UTC m=+55.708572912"
Sep 16 13:09:08.049962 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.049923    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-v86g5" podStartSLOduration=2.372849097 podStartE2EDuration="5.049910602s" totalImagesPullingTime="2.677061505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:08.048960546 +0000 UTC m=+55.725674982" watchObservedRunningTime="2026-09-16 13:09:08.049910602 +0000 UTC m=+55.726625036"
Sep 16 13:09:08.400538 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.400503    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p8kr9"]
Sep 16 13:09:08.403243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.402938    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.405433 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.405410    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 13:09:08.405573 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.405436    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 13:09:08.406949 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.406919    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dlvxs\""
Sep 16 13:09:08.415684 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.415663    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8kr9"]
Sep 16 13:09:08.476570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.476545    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.476705 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.476575    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-262nx\" (UniqueName: \"kubernetes.io/projected/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-api-access-262nx\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.476775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.476715    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a912322b-4b85-4e9e-857c-f18088f5e7ab-crio-socket\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.476775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.476761    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912322b-4b85-4e9e-857c-f18088f5e7ab-data-volume\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.476848 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.476815    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577321 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a912322b-4b85-4e9e-857c-f18088f5e7ab-crio-socket\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577347    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912322b-4b85-4e9e-857c-f18088f5e7ab-data-volume\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577554 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577431    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a912322b-4b85-4e9e-857c-f18088f5e7ab-crio-socket\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577554 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577485    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577554 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-262nx\" (UniqueName: \"kubernetes.io/projected/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-api-access-262nx\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.577554 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:08.577512    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:08.577776 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:08.577564    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:09.077547339 +0000 UTC m=+56.754261758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:08.577776 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:08.577621    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:09:08.577776 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:08.577660    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:09.077647764 +0000 UTC m=+56.754362177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : configmap "kube-rbac-proxy" not found
Sep 16 13:09:08.577776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.577690    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912322b-4b85-4e9e-857c-f18088f5e7ab-data-volume\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:08.586101 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:08.586079    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-262nx\" (UniqueName: \"kubernetes.io/projected/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-api-access-262nx\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:09.024230 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.024201    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/1.log"
Sep 16 13:09:09.024633 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.024618    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/0.log"
Sep 16 13:09:09.024677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.024651    2556 generic.go:356] "Generic (PLEG): container finished" podID="91d784c7-fff8-4c82-9f66-70abca2656e9" containerID="9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c" exitCode=255
Sep 16 13:09:09.024737 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.024723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-878hd" event={"ID":"91d784c7-fff8-4c82-9f66-70abca2656e9","Type":"ContainerDied","Data":"9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c"}
Sep 16 13:09:09.024793 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.024762    2556 scope.go:117] "RemoveContainer" containerID="1eefc87b9895abc99ed0b85c39e50f95bec06b053e20767747845e00b15b55be"
Sep 16 13:09:09.025012 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.024989    2556 scope.go:117] "RemoveContainer" containerID="9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c"
Sep 16 13:09:09.025209 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:09.025183    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-878hd_openshift-console-operator(91d784c7-fff8-4c82-9f66-70abca2656e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podUID="91d784c7-fff8-4c82-9f66-70abca2656e9"
Sep 16 13:09:09.082223 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.082201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:09.082315 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.082295    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:09.082365 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:09.082350    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:09.082421 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:09.082404    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:09:09.082472 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:09.082406    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:10.082388784 +0000 UTC m=+57.759103209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:09.082472 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:09.082464    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:10.082449015 +0000 UTC m=+57.759163428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : configmap "kube-rbac-proxy" not found
Sep 16 13:09:09.761629 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:09.761583    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-td4wf_0a4bdde9-8ef0-4089-9fdf-9414452b6494/dns-node-resolver/0.log"
Sep 16 13:09:10.028639 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.028563    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/1.log"
Sep 16 13:09:10.029020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.028928    2556 scope.go:117] "RemoveContainer" containerID="9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c"
Sep 16 13:09:10.029139 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:10.029109    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-878hd_openshift-console-operator(91d784c7-fff8-4c82-9f66-70abca2656e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podUID="91d784c7-fff8-4c82-9f66-70abca2656e9"
Sep 16 13:09:10.030136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.030114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt" event={"ID":"d420b4ec-b559-4e74-964a-1bacec5229e5","Type":"ContainerStarted","Data":"dcddc4730342121fc06b466a383d5d9dac562969505e0b00ac3830b289fe11ee"}
Sep 16 13:09:10.069505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.069466    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt" podStartSLOduration=1.73829257 podStartE2EDuration="4.069453393s" totalImagesPullingTime="2.331160823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:10.069127595 +0000 UTC m=+57.745842029" watchObservedRunningTime="2026-09-16 13:09:10.069453393 +0000 UTC m=+57.746167825"
Sep 16 13:09:10.091726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.091692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:10.091829 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:10.091772    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:10.091829 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:10.091817    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:12.091805257 +0000 UTC m=+59.768519669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:10.091949 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.091864    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:10.091949 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:10.091933    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:09:10.092053 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:10.091986    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:12.091967727 +0000 UTC m=+59.768682141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : configmap "kube-rbac-proxy" not found
Sep 16 13:09:10.560678 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:10.560651    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rfr7l_3a2b4109-e4a9-467b-8ec7-a71ca968b738/node-ca/0.log"
Sep 16 13:09:11.161582 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:11.161534    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dlp96_8455cf6b-b3c9-4c05-b62d-613ed31ba4f0/serve-healthcheck-canary/0.log"
Sep 16 13:09:11.402190 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:11.402158    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:11.402348 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.402301    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:11.402396 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.402363    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls podName:e1502655-4118-4947-86c7-0040ae5a3791 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:19.40234685 +0000 UTC m=+67.079061264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7bv2q" (UID: "e1502655-4118-4947-86c7-0040ae5a3791") : secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:11.502945 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:11.502876    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:11.503076 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.503032    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 13:09:11.503142 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:11.503078    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:11.503142 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.503095    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:19.503075847 +0000 UTC m=+67.179790263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : secret "router-metrics-certs-default" not found
Sep 16 13:09:11.503235 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.503162    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:19.503149383 +0000 UTC m=+67.179863796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : configmap references non-existent config key: service-ca.crt
Sep 16 13:09:11.704282 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:11.704253    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:11.704465 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.704442    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 13:09:11.704541 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:11.704515    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls podName:9daa4d5e-c75c-4953-bdbd-f288ecb9b654 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:19.704497088 +0000 UTC m=+67.381211504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-mzx6j" (UID: "9daa4d5e-c75c-4953-bdbd-f288ecb9b654") : secret "samples-operator-tls" not found
Sep 16 13:09:12.107034 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:12.107007    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:12.107155 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:12.107075    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:12.107155 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:12.107135    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:09:12.107228 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:12.107188    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:16.10717615 +0000 UTC m=+63.783890564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : configmap "kube-rbac-proxy" not found
Sep 16 13:09:12.107293 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:12.107274    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:12.107335 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:12.107329    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:16.107317406 +0000 UTC m=+63.784031824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:14.289079 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:14.289047    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:14.289496 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:14.289132    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:14.289496 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:14.289418    2556 scope.go:117] "RemoveContainer" containerID="9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c"
Sep 16 13:09:14.289564 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:14.289546    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-878hd_openshift-console-operator(91d784c7-fff8-4c82-9f66-70abca2656e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podUID="91d784c7-fff8-4c82-9f66-70abca2656e9"
Sep 16 13:09:14.425873 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:14.425845    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rsvv"
Sep 16 13:09:16.139438 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:16.139404    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:16.139891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:16.139476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:16.139891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:16.139559    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:16.139891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:16.139572    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:09:16.139891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:16.139645    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:24.139624313 +0000 UTC m=+71.816338744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:16.139891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:16.139661    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:24.139654187 +0000 UTC m=+71.816368600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : configmap "kube-rbac-proxy" not found
Sep 16 13:09:18.560461 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.560430    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.560480    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.560575    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.560606    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dc5678fd9-5qr4n: secret "image-registry-tls" not found
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.560581    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.560660    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls podName:24e6abeb-d83c-4f33-821c-e0348b063346 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:50.560645395 +0000 UTC m=+98.237359809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls") pod "image-registry-dc5678fd9-5qr4n" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346") : secret "image-registry-tls" not found
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.560673    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:09:18.560818 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.560721    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls podName:6c759f5e-28c4-4176-9f32-b75b43efc524 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:50.560706716 +0000 UTC m=+98.237421149 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls") pod "dns-default-hn46r" (UID: "6c759f5e-28c4-4176-9f32-b75b43efc524") : secret "dns-default-metrics-tls" not found
Sep 16 13:09:18.562945 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.562927    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 13:09:18.571683 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.571669    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 13:09:18.571763 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:18.571717    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs podName:c4c5e114-51f5-4670-85af-f57c14bc6c58 nodeName:}" failed. No retries permitted until 2026-09-16 13:10:22.571706106 +0000 UTC m=+130.248420519 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs") pod "network-metrics-daemon-jdkss" (UID: "c4c5e114-51f5-4670-85af-f57c14bc6c58") : secret "metrics-daemon-secret" not found
Sep 16 13:09:18.661540 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.661518    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:09:18.661635 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.661551    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:09:18.664000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.663982    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 13:09:18.664000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.663992    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 13:09:18.674207 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.674188    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694264fb-023d-4197-b834-514ec068625c-original-pull-secret\") pod \"global-pull-secret-syncer-2n6bp\" (UID: \"694264fb-023d-4197-b834-514ec068625c\") " pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:09:18.674296 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.674219    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 13:09:18.685110 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.685087    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jzvc\" (UniqueName: \"kubernetes.io/projected/94ffed1b-cdef-48f6-90c6-a3ce87d96c4b-kube-api-access-8jzvc\") pod \"network-check-target-jtk45\" (UID: \"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b\") " pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:09:18.903351 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.903330    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6bp"
Sep 16 13:09:18.909216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.909196    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sfm4x\""
Sep 16 13:09:18.917425 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:18.917404    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:09:19.171900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:19.171828    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2n6bp"]
Sep 16 13:09:19.175002 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:19.174968    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694264fb_023d_4197_b834_514ec068625c.slice/crio-b9ef7a20f0f37dd3002eccb8cb0541c00f5993789e820745cfc91669d29f6186 WatchSource:0}: Error finding container b9ef7a20f0f37dd3002eccb8cb0541c00f5993789e820745cfc91669d29f6186: Status 404 returned error can't find the container with id b9ef7a20f0f37dd3002eccb8cb0541c00f5993789e820745cfc91669d29f6186
Sep 16 13:09:19.200984 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:19.200958    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jtk45"]
Sep 16 13:09:19.203328 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:19.203305    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ffed1b_cdef_48f6_90c6_a3ce87d96c4b.slice/crio-0abae9e964a1827c944ed15ddf72198e1cd57d71f3253e169932707b14f54186 WatchSource:0}: Error finding container 0abae9e964a1827c944ed15ddf72198e1cd57d71f3253e169932707b14f54186: Status 404 returned error can't find the container with id 0abae9e964a1827c944ed15ddf72198e1cd57d71f3253e169932707b14f54186
Sep 16 13:09:19.468631 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:19.468542    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:19.468758 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.468715    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:19.468798 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.468773    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls podName:e1502655-4118-4947-86c7-0040ae5a3791 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:35.468758498 +0000 UTC m=+83.145472937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7bv2q" (UID: "e1502655-4118-4947-86c7-0040ae5a3791") : secret "cluster-monitoring-operator-tls" not found
Sep 16 13:09:19.569284 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:19.569256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:19.569591 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:19.569311    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:19.569591 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.569416    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:35.569394531 +0000 UTC m=+83.246108965 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : configmap references non-existent config key: service-ca.crt
Sep 16 13:09:19.569591 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.569445    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 13:09:19.569591 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.569504    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs podName:01e0e7b1-ef76-4103-99cf-588da4736933 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:35.569489526 +0000 UTC m=+83.246203946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs") pod "router-default-675764b9db-82s2n" (UID: "01e0e7b1-ef76-4103-99cf-588da4736933") : secret "router-metrics-certs-default" not found
Sep 16 13:09:19.770232 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:19.770162    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:19.770377 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.770321    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 13:09:19.770439 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:19.770391    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls podName:9daa4d5e-c75c-4953-bdbd-f288ecb9b654 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:35.770372059 +0000 UTC m=+83.447086472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-mzx6j" (UID: "9daa4d5e-c75c-4953-bdbd-f288ecb9b654") : secret "samples-operator-tls" not found
Sep 16 13:09:20.055952 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:20.055854    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jtk45" event={"ID":"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b","Type":"ContainerStarted","Data":"0abae9e964a1827c944ed15ddf72198e1cd57d71f3253e169932707b14f54186"}
Sep 16 13:09:20.057532 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:20.057499    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2n6bp" event={"ID":"694264fb-023d-4197-b834-514ec068625c","Type":"ContainerStarted","Data":"b9ef7a20f0f37dd3002eccb8cb0541c00f5993789e820745cfc91669d29f6186"}
Sep 16 13:09:24.209098 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:24.209065    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:24.209474 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:24.209156    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:24.209474 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:24.209263    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:24.209474 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:24.209312    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls podName:a912322b-4b85-4e9e-857c-f18088f5e7ab nodeName:}" failed. No retries permitted until 2026-09-16 13:09:40.20929965 +0000 UTC m=+87.886014063 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8kr9" (UID: "a912322b-4b85-4e9e-857c-f18088f5e7ab") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:09:24.209751 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:24.209693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a912322b-4b85-4e9e-857c-f18088f5e7ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:25.073670 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:25.073634    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jtk45" event={"ID":"94ffed1b-cdef-48f6-90c6-a3ce87d96c4b","Type":"ContainerStarted","Data":"88a2ce922b1d670a9077908aa34c9be639388063dbc346e433b86c6a518bad11"}
Sep 16 13:09:25.073951 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:25.073742    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:09:25.075228 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:25.075202    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2n6bp" event={"ID":"694264fb-023d-4197-b834-514ec068625c","Type":"ContainerStarted","Data":"514d2ec484d524e038120b4881798f2e105a94a325143cec46dcc0fb217ccb1c"}
Sep 16 13:09:25.095547 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:25.095500    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jtk45" podStartSLOduration=67.224484175 podStartE2EDuration="1m12.095487779s" totalImagesPullingTime="4.871003604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:25.094404011 +0000 UTC m=+72.771118476" watchObservedRunningTime="2026-09-16 13:09:25.095487779 +0000 UTC m=+72.772202260"
Sep 16 13:09:25.114388 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:25.114351    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2n6bp" podStartSLOduration=66.210079215 podStartE2EDuration="1m11.114339384s" totalImagesPullingTime="4.904260169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:25.114146978 +0000 UTC m=+72.790861412" watchObservedRunningTime="2026-09-16 13:09:25.114339384 +0000 UTC m=+72.791053819"
Sep 16 13:09:28.794348 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:28.794311    2556 scope.go:117] "RemoveContainer" containerID="9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c"
Sep 16 13:09:29.089576 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:29.089527    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:09:29.089987 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:29.089971    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/1.log"
Sep 16 13:09:29.090038 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:29.090004    2556 generic.go:356] "Generic (PLEG): container finished" podID="91d784c7-fff8-4c82-9f66-70abca2656e9" containerID="c56dfe7d613f5bc0318e7f3889e535061995baacdd931229ee8dd31c42413294" exitCode=255
Sep 16 13:09:29.090038 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:29.090030    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-878hd" event={"ID":"91d784c7-fff8-4c82-9f66-70abca2656e9","Type":"ContainerDied","Data":"c56dfe7d613f5bc0318e7f3889e535061995baacdd931229ee8dd31c42413294"}
Sep 16 13:09:29.090106 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:29.090072    2556 scope.go:117] "RemoveContainer" containerID="9c61a808dc03c5305bb3ac098347006d37677c456e4789826c605bac4a43dc2c"
Sep 16 13:09:29.090333 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:29.090317    2556 scope.go:117] "RemoveContainer" containerID="c56dfe7d613f5bc0318e7f3889e535061995baacdd931229ee8dd31c42413294"
Sep 16 13:09:29.090497 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:29.090478    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-878hd_openshift-console-operator(91d784c7-fff8-4c82-9f66-70abca2656e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podUID="91d784c7-fff8-4c82-9f66-70abca2656e9"
Sep 16 13:09:30.094535 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:30.094509    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:09:34.289213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:34.289183    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:34.289573 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:34.289230    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:34.289573 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:34.289461    2556 scope.go:117] "RemoveContainer" containerID="c56dfe7d613f5bc0318e7f3889e535061995baacdd931229ee8dd31c42413294"
Sep 16 13:09:34.289683 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:34.289587    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-878hd_openshift-console-operator(91d784c7-fff8-4c82-9f66-70abca2656e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podUID="91d784c7-fff8-4c82-9f66-70abca2656e9"
Sep 16 13:09:35.495381 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.495241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:35.497760 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.497736    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1502655-4118-4947-86c7-0040ae5a3791-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7bv2q\" (UID: \"e1502655-4118-4947-86c7-0040ae5a3791\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:35.596212 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.596187    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:35.596330 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.596235    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:35.596810 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.596788    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e0e7b1-ef76-4103-99cf-588da4736933-service-ca-bundle\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:35.598277 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.598259    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e0e7b1-ef76-4103-99cf-588da4736933-metrics-certs\") pod \"router-default-675764b9db-82s2n\" (UID: \"01e0e7b1-ef76-4103-99cf-588da4736933\") " pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:35.774991 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.774927    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-446mb\""
Sep 16 13:09:35.782753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.782735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"
Sep 16 13:09:35.797650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.797621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:35.799796 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.799776    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9daa4d5e-c75c-4953-bdbd-f288ecb9b654-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-mzx6j\" (UID: \"9daa4d5e-c75c-4953-bdbd-f288ecb9b654\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:35.809185 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.809168    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lzq8b\""
Sep 16 13:09:35.817193 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.817167    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"
Sep 16 13:09:35.877984 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.877962    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6fr7k\""
Sep 16 13:09:35.886311 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:35.886096    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:36.050473 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:36.050454    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q"]
Sep 16 13:09:36.052920 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:36.052893    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1502655_4118_4947_86c7_0040ae5a3791.slice/crio-de1c97bc80caef4aecd86952acd0db9a23c0538b8d241e6d91ce82ceb65ba3e8 WatchSource:0}: Error finding container de1c97bc80caef4aecd86952acd0db9a23c0538b8d241e6d91ce82ceb65ba3e8: Status 404 returned error can't find the container with id de1c97bc80caef4aecd86952acd0db9a23c0538b8d241e6d91ce82ceb65ba3e8
Sep 16 13:09:36.089237 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:36.089209    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j"]
Sep 16 13:09:36.115292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:36.115265    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q" event={"ID":"e1502655-4118-4947-86c7-0040ae5a3791","Type":"ContainerStarted","Data":"de1c97bc80caef4aecd86952acd0db9a23c0538b8d241e6d91ce82ceb65ba3e8"}
Sep 16 13:09:36.153586 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:36.153562    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-675764b9db-82s2n"]
Sep 16 13:09:36.156821 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:36.156795    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e0e7b1_ef76_4103_99cf_588da4736933.slice/crio-a8544235ac5407ae3eeac22f62df95ef9075e99385a5f3a13b131d4296410554 WatchSource:0}: Error finding container a8544235ac5407ae3eeac22f62df95ef9075e99385a5f3a13b131d4296410554: Status 404 returned error can't find the container with id a8544235ac5407ae3eeac22f62df95ef9075e99385a5f3a13b131d4296410554
Sep 16 13:09:37.119848 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:37.119806    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j" event={"ID":"9daa4d5e-c75c-4953-bdbd-f288ecb9b654","Type":"ContainerStarted","Data":"a49d87dac41748e6d19e81b539da0417c9df6c3135c335baa2687947339eaee5"}
Sep 16 13:09:37.121456 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:37.121430    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-675764b9db-82s2n" event={"ID":"01e0e7b1-ef76-4103-99cf-588da4736933","Type":"ContainerStarted","Data":"bc32646552befdd8fceed5e1450c940deaf272db813a42eb0b84ba6ca8b8a86b"}
Sep 16 13:09:37.121593 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:37.121470    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-675764b9db-82s2n" event={"ID":"01e0e7b1-ef76-4103-99cf-588da4736933","Type":"ContainerStarted","Data":"a8544235ac5407ae3eeac22f62df95ef9075e99385a5f3a13b131d4296410554"}
Sep 16 13:09:37.144105 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:37.144054    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-675764b9db-82s2n" podStartSLOduration=34.144036942 podStartE2EDuration="34.144036942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:37.14315861 +0000 UTC m=+84.819873046" watchObservedRunningTime="2026-09-16 13:09:37.144036942 +0000 UTC m=+84.820751376"
Sep 16 13:09:37.886408 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:37.886369    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:37.889373 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:37.889349    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:38.125680 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:38.125648    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:38.127224 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:38.127198    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-675764b9db-82s2n"
Sep 16 13:09:39.130659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:39.130619    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q" event={"ID":"e1502655-4118-4947-86c7-0040ae5a3791","Type":"ContainerStarted","Data":"4e89cc23eddfe8f26cfd3adbcc73643d49cabbfdec1566decde3e34a9180136d"}
Sep 16 13:09:39.132615 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:39.132562    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j" event={"ID":"9daa4d5e-c75c-4953-bdbd-f288ecb9b654","Type":"ContainerStarted","Data":"a3e57d1f355fd06face61c56bfeab0815431f5316f64039375f049398909e870"}
Sep 16 13:09:39.132740 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:39.132624    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j" event={"ID":"9daa4d5e-c75c-4953-bdbd-f288ecb9b654","Type":"ContainerStarted","Data":"b88880acef569739566cf0ea30d2d88f79e302abb1bba4fe22d29ec67891c3fd"}
Sep 16 13:09:39.146702 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:39.146658    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7bv2q" podStartSLOduration=33.498420986 podStartE2EDuration="36.146642551s" totalImagesPullingTime="2.648221565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:39.145822727 +0000 UTC m=+86.822537160" watchObservedRunningTime="2026-09-16 13:09:39.146642551 +0000 UTC m=+86.823357030"
Sep 16 13:09:39.161846 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:39.161806    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-mzx6j" podStartSLOduration=33.593176897 podStartE2EDuration="36.161785605s" totalImagesPullingTime="2.568608708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:39.160794068 +0000 UTC m=+86.837508506" watchObservedRunningTime="2026-09-16 13:09:39.161785605 +0000 UTC m=+86.838500083"
Sep 16 13:09:40.234137 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:40.234103    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:40.236437 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:40.236403    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a912322b-4b85-4e9e-857c-f18088f5e7ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8kr9\" (UID: \"a912322b-4b85-4e9e-857c-f18088f5e7ab\") " pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:40.515959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:40.515894    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dlvxs\""
Sep 16 13:09:40.523836 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:40.523815    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8kr9"
Sep 16 13:09:40.799860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:40.799783    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8kr9"]
Sep 16 13:09:40.801904 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:40.801877    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda912322b_4b85_4e9e_857c_f18088f5e7ab.slice/crio-7365d80ab089222bb754be736460708f4af165131e68d17de2d34071e32049db WatchSource:0}: Error finding container 7365d80ab089222bb754be736460708f4af165131e68d17de2d34071e32049db: Status 404 returned error can't find the container with id 7365d80ab089222bb754be736460708f4af165131e68d17de2d34071e32049db
Sep 16 13:09:41.139825 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:41.139793    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8kr9" event={"ID":"a912322b-4b85-4e9e-857c-f18088f5e7ab","Type":"ContainerStarted","Data":"a0221ca5053487c0a9e6ceab1180aa835c1964791bad5cdedbff700723de6ba1"}
Sep 16 13:09:41.139957 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:41.139872    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8kr9" event={"ID":"a912322b-4b85-4e9e-857c-f18088f5e7ab","Type":"ContainerStarted","Data":"7365d80ab089222bb754be736460708f4af165131e68d17de2d34071e32049db"}
Sep 16 13:09:42.145197 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:42.145164    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8kr9" event={"ID":"a912322b-4b85-4e9e-857c-f18088f5e7ab","Type":"ContainerStarted","Data":"ac89417e101bdd2efb1f02ca41cb506d2849624301e7602709d0d943bd456f49"}
Sep 16 13:09:44.154689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:44.154657    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8kr9" event={"ID":"a912322b-4b85-4e9e-857c-f18088f5e7ab","Type":"ContainerStarted","Data":"5e74021cb1c971ff842ae9b6fbcddfb416d71ab218acda82ed66c4051c599766"}
Sep 16 13:09:44.175183 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:44.175132    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p8kr9" podStartSLOduration=33.385427477 podStartE2EDuration="36.175119419s" totalImagesPullingTime="2.789691942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:09:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:44.17415734 +0000 UTC m=+91.850871775" watchObservedRunningTime="2026-09-16 13:09:44.175119419 +0000 UTC m=+91.851833853"
Sep 16 13:09:44.794971 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:44.794938    2556 scope.go:117] "RemoveContainer" containerID="c56dfe7d613f5bc0318e7f3889e535061995baacdd931229ee8dd31c42413294"
Sep 16 13:09:44.795150 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:44.795126    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-878hd_openshift-console-operator(91d784c7-fff8-4c82-9f66-70abca2656e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podUID="91d784c7-fff8-4c82-9f66-70abca2656e9"
Sep 16 13:09:45.624885 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.624859    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"]
Sep 16 13:09:45.627162 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.627144    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.630098 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.630079    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 13:09:45.630344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.630327    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 13:09:45.630498 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.630483    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 13:09:45.631144 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.631128    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 13:09:45.631416 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.631401    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 13:09:45.632444 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.632427    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 13:09:45.634365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.634233    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 13:09:45.648700 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.648680    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"]
Sep 16 13:09:45.780300 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.780274    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqsjb\" (UniqueName: \"kubernetes.io/projected/6551cfb9-f482-4b5b-8b44-d22cd588752a-kube-api-access-bqsjb\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.780390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.780339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-ca\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.780390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.780358    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-hub\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.780390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.780372    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6551cfb9-f482-4b5b-8b44-d22cd588752a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.780504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.780399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.780504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.780425    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881098 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881038    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqsjb\" (UniqueName: \"kubernetes.io/projected/6551cfb9-f482-4b5b-8b44-d22cd588752a-kube-api-access-bqsjb\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881212 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881097    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-ca\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881212 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881113    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-hub\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881212 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881128    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6551cfb9-f482-4b5b-8b44-d22cd588752a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881212 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881158    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881212 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881196    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.881925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.881899    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6551cfb9-f482-4b5b-8b44-d22cd588752a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.883617 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.883578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.883722 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.883667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-ca\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.883868 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.883847    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-hub\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.883925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.883912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6551cfb9-f482-4b5b-8b44-d22cd588752a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.893834 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.893814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqsjb\" (UniqueName: \"kubernetes.io/projected/6551cfb9-f482-4b5b-8b44-d22cd588752a-kube-api-access-bqsjb\") pod \"cluster-proxy-proxy-agent-f6cd8b778-292pb\" (UID: \"6551cfb9-f482-4b5b-8b44-d22cd588752a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:45.946308 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:45.946286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"
Sep 16 13:09:46.223112 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:46.223040    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb"]
Sep 16 13:09:46.225545 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:46.225517    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6551cfb9_f482_4b5b_8b44_d22cd588752a.slice/crio-555f479a6e33aa2f7f70947413bd085503d361f4e1d0749b6ffafadca675c103 WatchSource:0}: Error finding container 555f479a6e33aa2f7f70947413bd085503d361f4e1d0749b6ffafadca675c103: Status 404 returned error can't find the container with id 555f479a6e33aa2f7f70947413bd085503d361f4e1d0749b6ffafadca675c103
Sep 16 13:09:47.165524 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:47.165483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb" event={"ID":"6551cfb9-f482-4b5b-8b44-d22cd588752a","Type":"ContainerStarted","Data":"555f479a6e33aa2f7f70947413bd085503d361f4e1d0749b6ffafadca675c103"}
Sep 16 13:09:50.180605 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.180559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb" event={"ID":"6551cfb9-f482-4b5b-8b44-d22cd588752a","Type":"ContainerStarted","Data":"e2070d81fe1b58c9a1e72bf16ef0a853ff0d741db4ed4fd207a8156ef160a721"}
Sep 16 13:09:50.180925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.180617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb" event={"ID":"6551cfb9-f482-4b5b-8b44-d22cd588752a","Type":"ContainerStarted","Data":"f362e9db1eb97b6479f4ffd5d4fdd83f55c3cfc45b4d072bf19ba69b1c443229"}
Sep 16 13:09:50.180925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.180630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb" event={"ID":"6551cfb9-f482-4b5b-8b44-d22cd588752a","Type":"ContainerStarted","Data":"164bbe0a4da7c71735732a0076ff6478b64f9834e0cbf87b9b52ad1e249ff284"}
Sep 16 13:09:50.200507 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.200464    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cd8b778-292pb" podStartSLOduration=1.965341199 podStartE2EDuration="5.200449428s" totalImagesPullingTime="3.235108229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:50.199395292 +0000 UTC m=+97.876109726" watchObservedRunningTime="2026-09-16 13:09:50.200449428 +0000 UTC m=+97.877163862"
Sep 16 13:09:50.621942 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.621919    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:09:50.622056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.621997    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:09:50.624209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.624185    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c759f5e-28c4-4176-9f32-b75b43efc524-metrics-tls\") pod \"dns-default-hn46r\" (UID: \"6c759f5e-28c4-4176-9f32-b75b43efc524\") " pod="openshift-dns/dns-default-hn46r"
Sep 16 13:09:50.624299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.624270    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"image-registry-dc5678fd9-5qr4n\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") " pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:09:50.906786 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.906723    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lhrr5\""
Sep 16 13:09:50.914910 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.914892    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:09:50.921132 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.921112    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ftxst\""
Sep 16 13:09:50.928921 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:50.928902    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hn46r"
Sep 16 13:09:51.193376 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:51.193349    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dc5678fd9-5qr4n"]
Sep 16 13:09:51.195792 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:51.195760    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e6abeb_d83c_4f33_821c_e0348b063346.slice/crio-e507cbf45ae75fc7c359fd9bc1b9e58d38c392fb3fb636b36784d185b00ac89a WatchSource:0}: Error finding container e507cbf45ae75fc7c359fd9bc1b9e58d38c392fb3fb636b36784d185b00ac89a: Status 404 returned error can't find the container with id e507cbf45ae75fc7c359fd9bc1b9e58d38c392fb3fb636b36784d185b00ac89a
Sep 16 13:09:51.211283 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:51.211260    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hn46r"]
Sep 16 13:09:51.213899 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:51.213878    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c759f5e_28c4_4176_9f32_b75b43efc524.slice/crio-fc2a8a38d7c9e3d20c5666df1b1b2d58175710f81e1de438950a2fdd8ae3eb0e WatchSource:0}: Error finding container fc2a8a38d7c9e3d20c5666df1b1b2d58175710f81e1de438950a2fdd8ae3eb0e: Status 404 returned error can't find the container with id fc2a8a38d7c9e3d20c5666df1b1b2d58175710f81e1de438950a2fdd8ae3eb0e
Sep 16 13:09:52.188155 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:52.188114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hn46r" event={"ID":"6c759f5e-28c4-4176-9f32-b75b43efc524","Type":"ContainerStarted","Data":"fc2a8a38d7c9e3d20c5666df1b1b2d58175710f81e1de438950a2fdd8ae3eb0e"}
Sep 16 13:09:52.189472 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:52.189446    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n" event={"ID":"24e6abeb-d83c-4f33-821c-e0348b063346","Type":"ContainerStarted","Data":"1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860"}
Sep 16 13:09:52.189623 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:52.189484    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n" event={"ID":"24e6abeb-d83c-4f33-821c-e0348b063346","Type":"ContainerStarted","Data":"e507cbf45ae75fc7c359fd9bc1b9e58d38c392fb3fb636b36784d185b00ac89a"}
Sep 16 13:09:52.189623 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:52.189570    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:09:52.212054 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:52.212006    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n" podStartSLOduration=98.211990995 podStartE2EDuration="1m38.211990995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:52.210169359 +0000 UTC m=+99.886883804" watchObservedRunningTime="2026-09-16 13:09:52.211990995 +0000 UTC m=+99.888705429"
Sep 16 13:09:53.198820 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:53.195349    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hn46r" event={"ID":"6c759f5e-28c4-4176-9f32-b75b43efc524","Type":"ContainerStarted","Data":"f3befba5aeaeb2bc4c6fbed26de64171f81be9771f8d618c3fce3f3018d9db72"}
Sep 16 13:09:53.198820 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:53.195458    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hn46r" event={"ID":"6c759f5e-28c4-4176-9f32-b75b43efc524","Type":"ContainerStarted","Data":"c22030323d23bd6f3bfdb6436d4aabfbfe21cf3663b0670a74ed15f46ad7af0c"}
Sep 16 13:09:53.198820 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:53.195705    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hn46r"
Sep 16 13:09:53.213052 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:53.213008    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hn46r" podStartSLOduration=65.421060844 podStartE2EDuration="1m7.21299414s" totalImagesPullingTime="1.791933296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:53.212315348 +0000 UTC m=+100.889029805" watchObservedRunningTime="2026-09-16 13:09:53.21299414 +0000 UTC m=+100.889708553"
Sep 16 13:09:56.080686 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:56.080653    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jtk45"
Sep 16 13:09:57.794384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:57.794347    2556 scope.go:117] "RemoveContainer" containerID="c56dfe7d613f5bc0318e7f3889e535061995baacdd931229ee8dd31c42413294"
Sep 16 13:09:58.215269 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.215240    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:09:58.215457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.215296    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-878hd" event={"ID":"91d784c7-fff8-4c82-9f66-70abca2656e9","Type":"ContainerStarted","Data":"9d40f2b2c2f809424b694b0cc0ec1184c73c437c9345da1950acc312ed4d63e3"}
Sep 16 13:09:58.215584 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.215564    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:58.234384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.234340    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-878hd" podStartSLOduration=52.569994097 podStartE2EDuration="55.234325928s" totalImagesPullingTime="2.664331831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:09:58.233368714 +0000 UTC m=+105.910083174" watchObservedRunningTime="2026-09-16 13:09:58.234325928 +0000 UTC m=+105.911040362"
Sep 16 13:09:58.713338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.713288    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t9j4r"]
Sep 16 13:09:58.728740 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.728711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.731062 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.731040    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 13:09:58.731291 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.731275    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-djg9t\""
Sep 16 13:09:58.731374 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.731339    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 13:09:58.732029 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.732013    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 13:09:58.732086 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.732033    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 13:09:58.782932 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.782909    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fmtk\" (UniqueName: \"kubernetes.io/projected/b41f34de-bcd5-418c-893d-ba16e97f2c99-kube-api-access-4fmtk\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.782936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b41f34de-bcd5-418c-893d-ba16e97f2c99-metrics-client-ca\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.782980    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.783052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-sys\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.783119    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-tls\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.783148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-root\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.783172    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-wtmp\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.783210    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-textfile\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.783466 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.783233    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-accelerators-collector-config\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883611 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883571    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-sys\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-tls\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-root\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-wtmp\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883708    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-textfile\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883730    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-root\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:58.883745    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883733    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-accelerators-collector-config\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883787    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fmtk\" (UniqueName: \"kubernetes.io/projected/b41f34de-bcd5-418c-893d-ba16e97f2c99-kube-api-access-4fmtk\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:09:58.883810    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-tls podName:b41f34de-bcd5-418c-893d-ba16e97f2c99 nodeName:}" failed. No retries permitted until 2026-09-16 13:09:59.383795113 +0000 UTC m=+107.060509526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-tls") pod "node-exporter-t9j4r" (UID: "b41f34de-bcd5-418c-893d-ba16e97f2c99") : secret "node-exporter-tls" not found
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883820    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-wtmp\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.883976 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883836    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b41f34de-bcd5-418c-893d-ba16e97f2c99-metrics-client-ca\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.884454 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.883705    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b41f34de-bcd5-418c-893d-ba16e97f2c99-sys\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.884454 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.884225    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-textfile\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.884454 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.884346    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-accelerators-collector-config\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.884454 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.884398    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b41f34de-bcd5-418c-893d-ba16e97f2c99-metrics-client-ca\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.886278 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.886257    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:58.895584 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:58.895560    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fmtk\" (UniqueName: \"kubernetes.io/projected/b41f34de-bcd5-418c-893d-ba16e97f2c99-kube-api-access-4fmtk\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:59.204729 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:59.204703    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-878hd"
Sep 16 13:09:59.386975 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:59.386940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-tls\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:59.389082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:59.389064    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b41f34de-bcd5-418c-893d-ba16e97f2c99-node-exporter-tls\") pod \"node-exporter-t9j4r\" (UID: \"b41f34de-bcd5-418c-893d-ba16e97f2c99\") " pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:59.638136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:09:59.638102    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t9j4r"
Sep 16 13:09:59.648956 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:09:59.648923    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41f34de_bcd5_418c_893d_ba16e97f2c99.slice/crio-3cdfc659c870dd8a411a302d96c6c69edbf589ae786b0aa51bf69ef19759968c WatchSource:0}: Error finding container 3cdfc659c870dd8a411a302d96c6c69edbf589ae786b0aa51bf69ef19759968c: Status 404 returned error can't find the container with id 3cdfc659c870dd8a411a302d96c6c69edbf589ae786b0aa51bf69ef19759968c
Sep 16 13:10:00.226069 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:00.226035    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t9j4r" event={"ID":"b41f34de-bcd5-418c-893d-ba16e97f2c99","Type":"ContainerStarted","Data":"3cdfc659c870dd8a411a302d96c6c69edbf589ae786b0aa51bf69ef19759968c"}
Sep 16 13:10:01.231216 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:01.231175    2556 generic.go:356] "Generic (PLEG): container finished" podID="b41f34de-bcd5-418c-893d-ba16e97f2c99" containerID="e6a15fc6f74850807d1abac76cc2f09637451faf3c7e5e5a8e1432730c7a7aa1" exitCode=0
Sep 16 13:10:01.231719 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:01.231230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t9j4r" event={"ID":"b41f34de-bcd5-418c-893d-ba16e97f2c99","Type":"ContainerDied","Data":"e6a15fc6f74850807d1abac76cc2f09637451faf3c7e5e5a8e1432730c7a7aa1"}
Sep 16 13:10:02.240209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:02.240175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t9j4r" event={"ID":"b41f34de-bcd5-418c-893d-ba16e97f2c99","Type":"ContainerStarted","Data":"e4f0f76a73504cf5b1f3d6d3b1c2e29740d3a91e33d438cecffd94ab83106aec"}
Sep 16 13:10:02.240590 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:02.240227    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t9j4r" event={"ID":"b41f34de-bcd5-418c-893d-ba16e97f2c99","Type":"ContainerStarted","Data":"37e67ba0e4729b658c4cd3b2a9bb047e8441853a1c092caa2390b9c66ec9dacc"}
Sep 16 13:10:02.261855 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:02.261803    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t9j4r" podStartSLOduration=3.138898845 podStartE2EDuration="4.261788486s" totalImagesPullingTime="1.060633911s" totalInitContainerRuntime="62.25573ms" isStatefulPod=true podCreationTimestamp="2026-09-16 13:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:10:02.261359801 +0000 UTC m=+109.938074236" watchObservedRunningTime="2026-09-16 13:10:02.261788486 +0000 UTC m=+109.938502921"
Sep 16 13:10:03.200991 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:03.200962    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hn46r"
Sep 16 13:10:08.868071 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:08.868040    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dc5678fd9-5qr4n"]
Sep 16 13:10:08.872388 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:08.872365    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:10:22.659935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:22.659894    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:10:22.662257 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:22.662230    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c5e114-51f5-4670-85af-f57c14bc6c58-metrics-certs\") pod \"network-metrics-daemon-jdkss\" (UID: \"c4c5e114-51f5-4670-85af-f57c14bc6c58\") " pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:10:22.905064 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:22.905031    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f5rcg\""
Sep 16 13:10:22.912845 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:22.912794    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdkss"
Sep 16 13:10:23.235393 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:23.235304    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jdkss"]
Sep 16 13:10:23.237781 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:10:23.237747    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c5e114_51f5_4670_85af_f57c14bc6c58.slice/crio-1c67c3cf3d5a17d2f4da192833bb8920f098eef46196bf4102ee4a2131134048 WatchSource:0}: Error finding container 1c67c3cf3d5a17d2f4da192833bb8920f098eef46196bf4102ee4a2131134048: Status 404 returned error can't find the container with id 1c67c3cf3d5a17d2f4da192833bb8920f098eef46196bf4102ee4a2131134048
Sep 16 13:10:23.314370 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:23.314337    2556 generic.go:356] "Generic (PLEG): container finished" podID="1219ffb1-73d9-4ea2-a992-2d379213dc9f" containerID="735729bdae33c20ec56a6f6edb8e4132e002013daf20afe987a4aad747d594d3" exitCode=0
Sep 16 13:10:23.314537 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:23.314413    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-v86g5" event={"ID":"1219ffb1-73d9-4ea2-a992-2d379213dc9f","Type":"ContainerDied","Data":"735729bdae33c20ec56a6f6edb8e4132e002013daf20afe987a4aad747d594d3"}
Sep 16 13:10:23.314777 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:23.314759    2556 scope.go:117] "RemoveContainer" containerID="735729bdae33c20ec56a6f6edb8e4132e002013daf20afe987a4aad747d594d3"
Sep 16 13:10:23.315550 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:23.315531    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jdkss" event={"ID":"c4c5e114-51f5-4670-85af-f57c14bc6c58","Type":"ContainerStarted","Data":"1c67c3cf3d5a17d2f4da192833bb8920f098eef46196bf4102ee4a2131134048"}
Sep 16 13:10:24.320928 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:24.320852    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jdkss" event={"ID":"c4c5e114-51f5-4670-85af-f57c14bc6c58","Type":"ContainerStarted","Data":"6a60d7cc840f055ad96730914a5be2027ac064a7f648df4790fab4895b669b86"}
Sep 16 13:10:24.322689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:24.322666    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-v86g5" event={"ID":"1219ffb1-73d9-4ea2-a992-2d379213dc9f","Type":"ContainerStarted","Data":"2bf5b44675715e5f53b0adbddc7f1515787e7194b3bd91c1bfd57b4013cd53db"}
Sep 16 13:10:25.327951 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:25.327916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jdkss" event={"ID":"c4c5e114-51f5-4670-85af-f57c14bc6c58","Type":"ContainerStarted","Data":"17de1ebc8133b4aa4a71ca520c09053cf2f8819a3cfaaa1269abec88b6755261"}
Sep 16 13:10:25.343662 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:25.343619    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jdkss" podStartSLOduration=131.425703563 podStartE2EDuration="2m12.343591366s" totalImagesPullingTime="917.887803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:10:25.343577838 +0000 UTC m=+133.020292286" watchObservedRunningTime="2026-09-16 13:10:25.343591366 +0000 UTC m=+133.020305801"
Sep 16 13:10:33.886764 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:33.886723    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n" podUID="24e6abeb-d83c-4f33-821c-e0348b063346" containerName="registry" containerID="cri-o://1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860" gracePeriod=30
Sep 16 13:10:34.123332 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.123310    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:10:34.250096 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250036    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-trusted-ca\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250097    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250129    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-bound-sa-token\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250159    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-image-registry-private-configuration\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250190    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pgz8\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-kube-api-access-8pgz8\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250219    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-installation-pull-secrets\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250411    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-registry-certificates\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250456    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:10:34.250490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250464    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24e6abeb-d83c-4f33-821c-e0348b063346-ca-trust-extracted\") pod \"24e6abeb-d83c-4f33-821c-e0348b063346\" (UID: \"24e6abeb-d83c-4f33-821c-e0348b063346\") "
Sep 16 13:10:34.250918 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250896    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-trusted-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.251010 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.250988    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:10:34.252526 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.252459    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:10:34.252526 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.252463    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:10:34.252781 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.252750    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-kube-api-access-8pgz8" (OuterVolumeSpecName: "kube-api-access-8pgz8") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "kube-api-access-8pgz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:10:34.252869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.252814    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:10:34.252914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.252875    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:10:34.259409 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.259386    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24e6abeb-d83c-4f33-821c-e0348b063346-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "24e6abeb-d83c-4f33-821c-e0348b063346" (UID: "24e6abeb-d83c-4f33-821c-e0348b063346"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:10:34.351966 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.351933    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-image-registry-private-configuration\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.351966 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.351970    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pgz8\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-kube-api-access-8pgz8\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.352130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.351985    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24e6abeb-d83c-4f33-821c-e0348b063346-installation-pull-secrets\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.352130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.352000    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24e6abeb-d83c-4f33-821c-e0348b063346-registry-certificates\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.352130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.352013    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24e6abeb-d83c-4f33-821c-e0348b063346-ca-trust-extracted\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.352130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.352026    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-registry-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.352130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.352039    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24e6abeb-d83c-4f33-821c-e0348b063346-bound-sa-token\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:10:34.360168 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.360144    2556 generic.go:356] "Generic (PLEG): container finished" podID="24e6abeb-d83c-4f33-821c-e0348b063346" containerID="1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860" exitCode=0
Sep 16 13:10:34.360276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.360183    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n"
Sep 16 13:10:34.360276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.360208    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n" event={"ID":"24e6abeb-d83c-4f33-821c-e0348b063346","Type":"ContainerDied","Data":"1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860"}
Sep 16 13:10:34.360276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.360246    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc5678fd9-5qr4n" event={"ID":"24e6abeb-d83c-4f33-821c-e0348b063346","Type":"ContainerDied","Data":"e507cbf45ae75fc7c359fd9bc1b9e58d38c392fb3fb636b36784d185b00ac89a"}
Sep 16 13:10:34.360276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.360265    2556 scope.go:117] "RemoveContainer" containerID="1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860"
Sep 16 13:10:34.368415 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.368400    2556 scope.go:117] "RemoveContainer" containerID="1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860"
Sep 16 13:10:34.368698 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:10:34.368675    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860\": container with ID starting with 1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860 not found: ID does not exist" containerID="1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860"
Sep 16 13:10:34.368789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.368703    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860"} err="failed to get container status \"1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860\": rpc error: code = NotFound desc = could not find container \"1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860\": container with ID starting with 1ead668d0c76f088bbbc93e63f2222966cfe8dd4e5c9aa82d6ca6c397537d860 not found: ID does not exist"
Sep 16 13:10:34.380084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.380061    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dc5678fd9-5qr4n"]
Sep 16 13:10:34.389410 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.389379    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-dc5678fd9-5qr4n"]
Sep 16 13:10:34.798403 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:34.798373    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24e6abeb-d83c-4f33-821c-e0348b063346" path="/var/lib/kubelet/pods/24e6abeb-d83c-4f33-821c-e0348b063346/volumes"
Sep 16 13:10:36.370384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:36.370352    2556 generic.go:356] "Generic (PLEG): container finished" podID="d420b4ec-b559-4e74-964a-1bacec5229e5" containerID="dcddc4730342121fc06b466a383d5d9dac562969505e0b00ac3830b289fe11ee" exitCode=0
Sep 16 13:10:36.370831 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:36.370421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt" event={"ID":"d420b4ec-b559-4e74-964a-1bacec5229e5","Type":"ContainerDied","Data":"dcddc4730342121fc06b466a383d5d9dac562969505e0b00ac3830b289fe11ee"}
Sep 16 13:10:36.370831 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:36.370784    2556 scope.go:117] "RemoveContainer" containerID="dcddc4730342121fc06b466a383d5d9dac562969505e0b00ac3830b289fe11ee"
Sep 16 13:10:37.376209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:10:37.376175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gjlt" event={"ID":"d420b4ec-b559-4e74-964a-1bacec5229e5","Type":"ContainerStarted","Data":"b6498dfa65cfe8f115c8f4104212521e1aaf722d804ca7acb9d6e173ce628980"}
Sep 16 13:13:08.602316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.602281    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"]
Sep 16 13:13:08.602732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.602527    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24e6abeb-d83c-4f33-821c-e0348b063346" containerName="registry"
Sep 16 13:13:08.602732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.602538    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="24e6abeb-d83c-4f33-821c-e0348b063346" containerName="registry"
Sep 16 13:13:08.602732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.602610    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="24e6abeb-d83c-4f33-821c-e0348b063346" containerName="registry"
Sep 16 13:13:08.605282 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.605267    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.607432 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.607411    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 16 13:13:08.607538 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.607480    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:13:08.607629 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.607593    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-fn5gc\""
Sep 16 13:13:08.621000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.620979    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"]
Sep 16 13:13:08.784623 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.784578    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/399ad2a5-0ad5-4c66-a649-3a92414226b4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rz8b\" (UID: \"399ad2a5-0ad5-4c66-a649-3a92414226b4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.784623 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.784634    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx9zk\" (UniqueName: \"kubernetes.io/projected/399ad2a5-0ad5-4c66-a649-3a92414226b4-kube-api-access-cx9zk\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rz8b\" (UID: \"399ad2a5-0ad5-4c66-a649-3a92414226b4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.885582 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.885553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/399ad2a5-0ad5-4c66-a649-3a92414226b4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rz8b\" (UID: \"399ad2a5-0ad5-4c66-a649-3a92414226b4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.885773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.885588    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9zk\" (UniqueName: \"kubernetes.io/projected/399ad2a5-0ad5-4c66-a649-3a92414226b4-kube-api-access-cx9zk\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rz8b\" (UID: \"399ad2a5-0ad5-4c66-a649-3a92414226b4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.885981 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.885961    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/399ad2a5-0ad5-4c66-a649-3a92414226b4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rz8b\" (UID: \"399ad2a5-0ad5-4c66-a649-3a92414226b4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.893309 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.893280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx9zk\" (UniqueName: \"kubernetes.io/projected/399ad2a5-0ad5-4c66-a649-3a92414226b4-kube-api-access-cx9zk\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rz8b\" (UID: \"399ad2a5-0ad5-4c66-a649-3a92414226b4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:08.913344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:08.913319    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"
Sep 16 13:13:09.190857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:09.190780    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b"]
Sep 16 13:13:09.194115 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:13:09.194087    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399ad2a5_0ad5_4c66_a649_3a92414226b4.slice/crio-6a026e8b65ac549ea8d42284dd69e4f84f4b46b58bdfb2900b06d8729c43c5e5 WatchSource:0}: Error finding container 6a026e8b65ac549ea8d42284dd69e4f84f4b46b58bdfb2900b06d8729c43c5e5: Status 404 returned error can't find the container with id 6a026e8b65ac549ea8d42284dd69e4f84f4b46b58bdfb2900b06d8729c43c5e5
Sep 16 13:13:09.917213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:09.917179    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b" event={"ID":"399ad2a5-0ad5-4c66-a649-3a92414226b4","Type":"ContainerStarted","Data":"6a026e8b65ac549ea8d42284dd69e4f84f4b46b58bdfb2900b06d8729c43c5e5"}
Sep 16 13:13:12.742342 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:12.742309    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:13:12.744901 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:12.744875    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:13:12.763121 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:12.763097    2556 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 13:13:12.930415 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:12.930374    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b" event={"ID":"399ad2a5-0ad5-4c66-a649-3a92414226b4","Type":"ContainerStarted","Data":"098a1167791d867528f29d199a9e769ddae745e07039238fb0b3010d883e8b49"}
Sep 16 13:13:12.968142 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:12.968097    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rz8b" podStartSLOduration=2.154524133 podStartE2EDuration="4.968083131s" totalImagesPullingTime="2.813558998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:13:12.967358363 +0000 UTC m=+300.644072798" watchObservedRunningTime="2026-09-16 13:13:12.968083131 +0000 UTC m=+300.644797565"
Sep 16 13:13:51.323105 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.323069    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"]
Sep 16 13:13:51.325307 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.325289    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.334746 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.334726    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 13:13:51.336419 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.336399    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 13:13:51.336519 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.336443    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 13:13:51.336519 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.336474    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 13:13:51.336856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.336835    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qxzqt\""
Sep 16 13:13:51.359035 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.359010    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"]
Sep 16 13:13:51.382961 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.382933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/183d736d-9ec9-4501-8d34-62c2d5bd81d5-manifests\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.383050 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.382971    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/183d736d-9ec9-4501-8d34-62c2d5bd81d5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.383050 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.382987    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/183d736d-9ec9-4501-8d34-62c2d5bd81d5-webhook-cert\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.383050 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.383028    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6kn4\" (UniqueName: \"kubernetes.io/projected/183d736d-9ec9-4501-8d34-62c2d5bd81d5-kube-api-access-x6kn4\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.383154 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.383066    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/183d736d-9ec9-4501-8d34-62c2d5bd81d5-tmp\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.483548 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.483515    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/183d736d-9ec9-4501-8d34-62c2d5bd81d5-manifests\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.483730 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.483561    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/183d736d-9ec9-4501-8d34-62c2d5bd81d5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.483730 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.483586    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/183d736d-9ec9-4501-8d34-62c2d5bd81d5-webhook-cert\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.483730 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.483636    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6kn4\" (UniqueName: \"kubernetes.io/projected/183d736d-9ec9-4501-8d34-62c2d5bd81d5-kube-api-access-x6kn4\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.483730 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.483663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/183d736d-9ec9-4501-8d34-62c2d5bd81d5-tmp\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.484096 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.484066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/183d736d-9ec9-4501-8d34-62c2d5bd81d5-manifests\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.485973 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.485954    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/183d736d-9ec9-4501-8d34-62c2d5bd81d5-tmp\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.486119 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.486097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/183d736d-9ec9-4501-8d34-62c2d5bd81d5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.486181 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.486166    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/183d736d-9ec9-4501-8d34-62c2d5bd81d5-webhook-cert\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.498474 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.498454    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6kn4\" (UniqueName: \"kubernetes.io/projected/183d736d-9ec9-4501-8d34-62c2d5bd81d5-kube-api-access-x6kn4\") pod \"opendatahub-operator-controller-manager-694ff5d95c-p4mhm\" (UID: \"183d736d-9ec9-4501-8d34-62c2d5bd81d5\") " pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.634938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.634912    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:51.923802 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.923777    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"]
Sep 16 13:13:51.926499 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:13:51.926472    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183d736d_9ec9_4501_8d34_62c2d5bd81d5.slice/crio-30c15b97c9b5b9ca74cbb724e2ef1f6e61d65d778389463f7cca733093a7900d WatchSource:0}: Error finding container 30c15b97c9b5b9ca74cbb724e2ef1f6e61d65d778389463f7cca733093a7900d: Status 404 returned error can't find the container with id 30c15b97c9b5b9ca74cbb724e2ef1f6e61d65d778389463f7cca733093a7900d
Sep 16 13:13:51.928250 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:51.928234    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:13:52.063876 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:52.063850    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm" event={"ID":"183d736d-9ec9-4501-8d34-62c2d5bd81d5","Type":"ContainerStarted","Data":"30c15b97c9b5b9ca74cbb724e2ef1f6e61d65d778389463f7cca733093a7900d"}
Sep 16 13:13:56.084933 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:56.084895    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm" event={"ID":"183d736d-9ec9-4501-8d34-62c2d5bd81d5","Type":"ContainerStarted","Data":"f56f431053f40d0ef8fb92a9fec991f149ffc5434b5a24b369603e745a3de6f5"}
Sep 16 13:13:57.091030 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:57.090998    2556 generic.go:356] "Generic (PLEG): container finished" podID="183d736d-9ec9-4501-8d34-62c2d5bd81d5" containerID="f56f431053f40d0ef8fb92a9fec991f149ffc5434b5a24b369603e745a3de6f5" exitCode=0
Sep 16 13:13:57.091399 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:57.091077    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm" event={"ID":"183d736d-9ec9-4501-8d34-62c2d5bd81d5","Type":"ContainerDied","Data":"f56f431053f40d0ef8fb92a9fec991f149ffc5434b5a24b369603e745a3de6f5"}
Sep 16 13:13:58.097113 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:58.097086    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm" event={"ID":"183d736d-9ec9-4501-8d34-62c2d5bd81d5","Type":"ContainerStarted","Data":"85d65cea8e4ed44f8b8f845e10804c3ba3c5c9fa0f2eb7cd04dfa8989d454c52"}
Sep 16 13:13:58.097446 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:58.097217    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:13:58.120149 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:13:58.120102    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm" podStartSLOduration=2.522366371 podStartE2EDuration="7.120087114s" totalImagesPullingTime="3.532441063s" totalInitContainerRuntime="1.06527968s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:13:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:13:58.118964535 +0000 UTC m=+345.795678982" watchObservedRunningTime="2026-09-16 13:13:58.120087114 +0000 UTC m=+345.796801548"
Sep 16 13:14:09.104203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:09.104128    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-694ff5d95c-p4mhm"
Sep 16 13:14:11.388166 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.388136    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"]
Sep 16 13:14:11.393957 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.393935    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.396916 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.396894    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 13:14:11.397014 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.396897    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 13:14:11.422126 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.422105    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"]
Sep 16 13:14:11.519668 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.519642    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tmp\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.519773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.519701    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl2bm\" (UniqueName: \"kubernetes.io/projected/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-kube-api-access-xl2bm\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.519773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.519759    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tls-certs\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.620979 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.620939    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tmp\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.621131 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.621009    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2bm\" (UniqueName: \"kubernetes.io/projected/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-kube-api-access-xl2bm\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.621131 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.621048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tls-certs\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.621202 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:14:11.621167    2556 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 16 13:14:11.621249 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:14:11.621238    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tls-certs podName:cd97e5ad-545f-4bee-b1ab-d9eca1b81626 nodeName:}" failed. No retries permitted until 2026-09-16 13:14:12.121217012 +0000 UTC m=+359.797931432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tls-certs") pod "kube-auth-proxy-6445dbdc5b-5z7fn" (UID: "cd97e5ad-545f-4bee-b1ab-d9eca1b81626") : secret "kube-auth-proxy-tls" not found
Sep 16 13:14:11.623228 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.623207    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tmp\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:11.633717 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:11.633697    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl2bm\" (UniqueName: \"kubernetes.io/projected/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-kube-api-access-xl2bm\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:12.124117 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:12.124087    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tls-certs\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:12.126469 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:12.126449    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd97e5ad-545f-4bee-b1ab-d9eca1b81626-tls-certs\") pod \"kube-auth-proxy-6445dbdc5b-5z7fn\" (UID: \"cd97e5ad-545f-4bee-b1ab-d9eca1b81626\") " pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:12.303281 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:12.303239    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"
Sep 16 13:14:12.602078 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:12.602055    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn"]
Sep 16 13:14:12.604166 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:14:12.604138    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd97e5ad_545f_4bee_b1ab_d9eca1b81626.slice/crio-de376511da7ad4b057aa8595d37f46be0572d28eaef6c0424e8b9ff44bfcb443 WatchSource:0}: Error finding container de376511da7ad4b057aa8595d37f46be0572d28eaef6c0424e8b9ff44bfcb443: Status 404 returned error can't find the container with id de376511da7ad4b057aa8595d37f46be0572d28eaef6c0424e8b9ff44bfcb443
Sep 16 13:14:13.058290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.058258    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"]
Sep 16 13:14:13.061342 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.061325    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.063738 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.063717    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 13:14:13.063845 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.063724    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 13:14:13.065113 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.065086    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 13:14:13.065215 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.065133    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2znts\""
Sep 16 13:14:13.065215 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.065092    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 13:14:13.086015 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.085995    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"]
Sep 16 13:14:13.134299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134276    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.134415 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134306    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.134415 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fe1b99-f08a-488d-9db1-2f2d7522be81-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.134415 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134347    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sppjl\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-kube-api-access-sppjl\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.134579 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134421    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.134579 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.134579 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.134513    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.155674 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.155648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn" event={"ID":"cd97e5ad-545f-4bee-b1ab-d9eca1b81626","Type":"ContainerStarted","Data":"de376511da7ad4b057aa8595d37f46be0572d28eaef6c0424e8b9ff44bfcb443"}
Sep 16 13:14:13.235339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235305    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.235339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235342    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.235521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235368    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fe1b99-f08a-488d-9db1-2f2d7522be81-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.235521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sppjl\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-kube-api-access-sppjl\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.235521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235418    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.235521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235455    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.235521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.235475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.236234 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.236207    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.237933 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.237912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.238023 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.237912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.238023 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.237984    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.238308 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.238282    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fe1b99-f08a-488d-9db1-2f2d7522be81-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.246509 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.246485    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sppjl\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-kube-api-access-sppjl\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.247036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.247012    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kqzxb\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.370247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.370223    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:13.641392 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:13.641335    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"]
Sep 16 13:14:13.644560 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:14:13.644527    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72fe1b99_f08a_488d_9db1_2f2d7522be81.slice/crio-0bbbaa77c082223e1d2309169f85f9df659a17d440ca661f168dfb8c2484871f WatchSource:0}: Error finding container 0bbbaa77c082223e1d2309169f85f9df659a17d440ca661f168dfb8c2484871f: Status 404 returned error can't find the container with id 0bbbaa77c082223e1d2309169f85f9df659a17d440ca661f168dfb8c2484871f
Sep 16 13:14:14.162421 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:14.162380    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" event={"ID":"72fe1b99-f08a-488d-9db1-2f2d7522be81","Type":"ContainerStarted","Data":"0bbbaa77c082223e1d2309169f85f9df659a17d440ca661f168dfb8c2484871f"}
Sep 16 13:14:15.583421 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:15.583393    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 13:14:16.172038 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:16.172002    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn" event={"ID":"cd97e5ad-545f-4bee-b1ab-d9eca1b81626","Type":"ContainerStarted","Data":"0e0bda060daac229322734cd68c39c7fb5469ec984c22ec89fe7ba9e87f574a6"}
Sep 16 13:14:16.189929 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:16.189877    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6445dbdc5b-5z7fn" podStartSLOduration=2.214748306 podStartE2EDuration="5.189855775s" totalImagesPullingTime="2.975107469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:14:16.18914192 +0000 UTC m=+363.865856378" watchObservedRunningTime="2026-09-16 13:14:16.189855775 +0000 UTC m=+363.866570211"
Sep 16 13:14:17.368887 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:17.368829    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 13:14:17.369247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:17.368933    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 13:14:18.182052 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:18.182016    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" event={"ID":"72fe1b99-f08a-488d-9db1-2f2d7522be81","Type":"ContainerStarted","Data":"e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39"}
Sep 16 13:14:18.182222 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:18.182068    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:18.183732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:18.183700    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-kqzxb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 13:14:18.183847 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:18.183760    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" podUID="72fe1b99-f08a-488d-9db1-2f2d7522be81" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 13:14:18.201066 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:18.201016    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" podStartSLOduration=1.481298349 podStartE2EDuration="5.201000083s" totalImagesPullingTime="3.719701734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:14:18.198911909 +0000 UTC m=+365.875626355" watchObservedRunningTime="2026-09-16 13:14:18.201000083 +0000 UTC m=+365.877714517"
Sep 16 13:14:19.187226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:19.187195    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:29.751484 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.751448    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"]
Sep 16 13:14:29.755042 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.755027    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.758582 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.758556    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 13:14:29.758777 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.758762    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 13:14:29.760025 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.760009    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 13:14:29.760100 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.760015    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:14:29.760290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.760265    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2g85n\""
Sep 16 13:14:29.760405 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.760352    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 13:14:29.784741 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.784720    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"]
Sep 16 13:14:29.870874 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.870842    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhzhl\" (UniqueName: \"kubernetes.io/projected/37d449d9-d828-4324-b6dd-6f71c54b7176-kube-api-access-xhzhl\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.871008 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.870885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37d449d9-d828-4324-b6dd-6f71c54b7176-manager-config\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.871008 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.870909    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d449d9-d828-4324-b6dd-6f71c54b7176-cert\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.871008 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.870937    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37d449d9-d828-4324-b6dd-6f71c54b7176-metrics-cert\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.971845 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.971802    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzhl\" (UniqueName: \"kubernetes.io/projected/37d449d9-d828-4324-b6dd-6f71c54b7176-kube-api-access-xhzhl\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.972020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.971873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37d449d9-d828-4324-b6dd-6f71c54b7176-manager-config\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.972020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.971907    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d449d9-d828-4324-b6dd-6f71c54b7176-cert\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.972020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.971926    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37d449d9-d828-4324-b6dd-6f71c54b7176-metrics-cert\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.972607 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.972567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37d449d9-d828-4324-b6dd-6f71c54b7176-manager-config\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.974281 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.974257    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d449d9-d828-4324-b6dd-6f71c54b7176-cert\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.974375 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.974294    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37d449d9-d828-4324-b6dd-6f71c54b7176-metrics-cert\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:29.980124 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:29.980101    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhzhl\" (UniqueName: \"kubernetes.io/projected/37d449d9-d828-4324-b6dd-6f71c54b7176-kube-api-access-xhzhl\") pod \"lws-controller-manager-5b97dbbc45-ntnmf\" (UID: \"37d449d9-d828-4324-b6dd-6f71c54b7176\") " pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:30.063832 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:30.063779    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:30.345018 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:30.344943    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"]
Sep 16 13:14:30.347686 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:14:30.347654    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d449d9_d828_4324_b6dd_6f71c54b7176.slice/crio-126218bf53c844dcb027229421ee4eca8e23763a90089d5d38a4605e956e576f WatchSource:0}: Error finding container 126218bf53c844dcb027229421ee4eca8e23763a90089d5d38a4605e956e576f: Status 404 returned error can't find the container with id 126218bf53c844dcb027229421ee4eca8e23763a90089d5d38a4605e956e576f
Sep 16 13:14:31.233925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:31.233888    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf" event={"ID":"37d449d9-d828-4324-b6dd-6f71c54b7176","Type":"ContainerStarted","Data":"126218bf53c844dcb027229421ee4eca8e23763a90089d5d38a4605e956e576f"}
Sep 16 13:14:32.978618 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:32.978571    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"]
Sep 16 13:14:32.978971 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:32.978776    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" podUID="72fe1b99-f08a-488d-9db1-2f2d7522be81" containerName="discovery" containerID="cri-o://e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39" gracePeriod=30
Sep 16 13:14:33.223885 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.223861    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:33.243558 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.243493    2556 generic.go:356] "Generic (PLEG): container finished" podID="72fe1b99-f08a-488d-9db1-2f2d7522be81" containerID="e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39" exitCode=0
Sep 16 13:14:33.243558 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.243529    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"
Sep 16 13:14:33.243725 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.243572    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" event={"ID":"72fe1b99-f08a-488d-9db1-2f2d7522be81","Type":"ContainerDied","Data":"e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39"}
Sep 16 13:14:33.243725 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.243645    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb" event={"ID":"72fe1b99-f08a-488d-9db1-2f2d7522be81","Type":"ContainerDied","Data":"0bbbaa77c082223e1d2309169f85f9df659a17d440ca661f168dfb8c2484871f"}
Sep 16 13:14:33.243725 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.243677    2556 scope.go:117] "RemoveContainer" containerID="e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39"
Sep 16 13:14:33.245207 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.245186    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf" event={"ID":"37d449d9-d828-4324-b6dd-6f71c54b7176","Type":"ContainerStarted","Data":"a675ef0cfe4a8a418048a985254abaaa8cb51051080d291194638294eabda401"}
Sep 16 13:14:33.245291 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.245274    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:14:33.251305 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.251289    2556 scope.go:117] "RemoveContainer" containerID="e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39"
Sep 16 13:14:33.251569 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:14:33.251551    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39\": container with ID starting with e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39 not found: ID does not exist" containerID="e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39"
Sep 16 13:14:33.251645 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.251580    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39"} err="failed to get container status \"e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39\": rpc error: code = NotFound desc = could not find container \"e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39\": container with ID starting with e6d4522facbed61c62faf7150ca8b503c5eeb726cce3accf7500b8d9926a7c39 not found: ID does not exist"
Sep 16 13:14:33.400461 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400432    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-cacerts\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.400589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400478    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sppjl\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-kube-api-access-sppjl\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.400589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400503    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-dns-cert\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.400589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400527    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-token\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.400589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400550    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fe1b99-f08a-488d-9db1-2f2d7522be81-local-certs\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.400845 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400633    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-ca-configmap\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.400845 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.400692    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-kubeconfig\") pod \"72fe1b99-f08a-488d-9db1-2f2d7522be81\" (UID: \"72fe1b99-f08a-488d-9db1-2f2d7522be81\") "
Sep 16 13:14:33.401080 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.401024    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:14:33.401862 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.401837    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-ca-configmap\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.403013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.402987    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-cacerts" (OuterVolumeSpecName: "cacerts") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:14:33.403112 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.403004    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:14:33.403273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.403248    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:14:33.403273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.403260    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-kube-api-access-sppjl" (OuterVolumeSpecName: "kube-api-access-sppjl") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "kube-api-access-sppjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:14:33.403392 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.403267    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fe1b99-f08a-488d-9db1-2f2d7522be81-local-certs" (OuterVolumeSpecName: "local-certs") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:14:33.403392 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.403282    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-token" (OuterVolumeSpecName: "istio-token") pod "72fe1b99-f08a-488d-9db1-2f2d7522be81" (UID: "72fe1b99-f08a-488d-9db1-2f2d7522be81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:14:33.502541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.502489    2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-cacerts\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.502541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.502509    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sppjl\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-kube-api-access-sppjl\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.502541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.502519    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-csr-dns-cert\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.502541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.502528    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-token\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.502541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.502537    2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fe1b99-f08a-488d-9db1-2f2d7522be81-local-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.502541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.502544    2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72fe1b99-f08a-488d-9db1-2f2d7522be81-istio-kubeconfig\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:14:33.563924 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.563870    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf" podStartSLOduration=1.892019249 podStartE2EDuration="4.563854044s" totalImagesPullingTime="2.671834795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:14:33.300982126 +0000 UTC m=+380.977696562" watchObservedRunningTime="2026-09-16 13:14:33.563854044 +0000 UTC m=+381.240568478"
Sep 16 13:14:33.564747 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.564730    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"]
Sep 16 13:14:33.569378 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:33.569354    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kqzxb"]
Sep 16 13:14:34.801160 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:34.801123    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72fe1b99-f08a-488d-9db1-2f2d7522be81" path="/var/lib/kubelet/pods/72fe1b99-f08a-488d-9db1-2f2d7522be81/volumes"
Sep 16 13:14:44.251650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:14:44.251611    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b97dbbc45-ntnmf"
Sep 16 13:15:57.865470 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.865436    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"]
Sep 16 13:15:57.865935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.865734    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72fe1b99-f08a-488d-9db1-2f2d7522be81" containerName="discovery"
Sep 16 13:15:57.865935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.865747    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="72fe1b99-f08a-488d-9db1-2f2d7522be81" containerName="discovery"
Sep 16 13:15:57.865935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.865801    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="72fe1b99-f08a-488d-9db1-2f2d7522be81" containerName="discovery"
Sep 16 13:15:57.867769 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.867749    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:57.870142 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.870119    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 13:15:57.870954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.870929    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 13:15:57.871060 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.870930    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 13:15:57.871263 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.871248    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hkwhd\""
Sep 16 13:15:57.871323 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.871249    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 13:15:57.876896 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:57.876875    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"]
Sep 16 13:15:58.010379 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.010346    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6631883-c889-456c-864d-996e1b6da645-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.010541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.010442    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9zvb\" (UniqueName: \"kubernetes.io/projected/b6631883-c889-456c-864d-996e1b6da645-kube-api-access-w9zvb\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.010541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.010469    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6631883-c889-456c-864d-996e1b6da645-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.111578 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.111550    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6631883-c889-456c-864d-996e1b6da645-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.111725 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.111609    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6631883-c889-456c-864d-996e1b6da645-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.111725 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.111696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9zvb\" (UniqueName: \"kubernetes.io/projected/b6631883-c889-456c-864d-996e1b6da645-kube-api-access-w9zvb\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.112345 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.112322    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6631883-c889-456c-864d-996e1b6da645-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.114056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.114034    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6631883-c889-456c-864d-996e1b6da645-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.122738 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.122714    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9zvb\" (UniqueName: \"kubernetes.io/projected/b6631883-c889-456c-864d-996e1b6da645-kube-api-access-w9zvb\") pod \"kuadrant-console-plugin-fbdd4f445-9npcm\" (UID: \"b6631883-c889-456c-864d-996e1b6da645\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.177066 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.177045    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"
Sep 16 13:15:58.447355 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.447274    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm"]
Sep 16 13:15:58.449981 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:15:58.449949    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6631883_c889_456c_864d_996e1b6da645.slice/crio-f931b9c618ff61091a30004b4d5cefb75cf42928dc653579e9509df084c66c58 WatchSource:0}: Error finding container f931b9c618ff61091a30004b4d5cefb75cf42928dc653579e9509df084c66c58: Status 404 returned error can't find the container with id f931b9c618ff61091a30004b4d5cefb75cf42928dc653579e9509df084c66c58
Sep 16 13:15:58.589290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:15:58.589261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm" event={"ID":"b6631883-c889-456c-864d-996e1b6da645","Type":"ContainerStarted","Data":"f931b9c618ff61091a30004b4d5cefb75cf42928dc653579e9509df084c66c58"}
Sep 16 13:16:03.617481 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:03.617396    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm" event={"ID":"b6631883-c889-456c-864d-996e1b6da645","Type":"ContainerStarted","Data":"86e979a0b21a973dc5621cac90a353de819afffdaad280f70b3d392b80e41626"}
Sep 16 13:16:03.633387 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:03.633340    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9npcm" podStartSLOduration=1.925532333 podStartE2EDuration="6.63332415s" totalImagesPullingTime="4.707791817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:16:03.632112831 +0000 UTC m=+471.308827267" watchObservedRunningTime="2026-09-16 13:16:03.63332415 +0000 UTC m=+471.310038585"
Sep 16 13:16:48.052373 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.052340    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:16:48.054851 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.054835    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.057065 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.057036    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 13:16:48.068177 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.068155    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:16:48.098273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.098247    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:16:48.179455 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.179435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d517954a-0f70-4689-bf72-edc01d46b1b1-config-file\") pod \"limitador-limitador-84994f7ffc-tfhpd\" (UID: \"d517954a-0f70-4689-bf72-edc01d46b1b1\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.179561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.179477    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lw79\" (UniqueName: \"kubernetes.io/projected/d517954a-0f70-4689-bf72-edc01d46b1b1-kube-api-access-2lw79\") pod \"limitador-limitador-84994f7ffc-tfhpd\" (UID: \"d517954a-0f70-4689-bf72-edc01d46b1b1\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.279783 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.279762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d517954a-0f70-4689-bf72-edc01d46b1b1-config-file\") pod \"limitador-limitador-84994f7ffc-tfhpd\" (UID: \"d517954a-0f70-4689-bf72-edc01d46b1b1\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.279885 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.279810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lw79\" (UniqueName: \"kubernetes.io/projected/d517954a-0f70-4689-bf72-edc01d46b1b1-kube-api-access-2lw79\") pod \"limitador-limitador-84994f7ffc-tfhpd\" (UID: \"d517954a-0f70-4689-bf72-edc01d46b1b1\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.280357 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.280340    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d517954a-0f70-4689-bf72-edc01d46b1b1-config-file\") pod \"limitador-limitador-84994f7ffc-tfhpd\" (UID: \"d517954a-0f70-4689-bf72-edc01d46b1b1\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.290676 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.290654    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lw79\" (UniqueName: \"kubernetes.io/projected/d517954a-0f70-4689-bf72-edc01d46b1b1-kube-api-access-2lw79\") pod \"limitador-limitador-84994f7ffc-tfhpd\" (UID: \"d517954a-0f70-4689-bf72-edc01d46b1b1\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.364207 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.364159    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:48.635394 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.635365    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-rqgwl"]
Sep 16 13:16:48.638320 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.638297    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:16:48.638439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.638417    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:16:48.638742 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:16:48.638719    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd517954a_0f70_4689_bf72_edc01d46b1b1.slice/crio-7c69763e3bd0dc750ea8b01abf0b3e842823495cf2211d92b5a88e57c70e6bf1 WatchSource:0}: Error finding container 7c69763e3bd0dc750ea8b01abf0b3e842823495cf2211d92b5a88e57c70e6bf1: Status 404 returned error can't find the container with id 7c69763e3bd0dc750ea8b01abf0b3e842823495cf2211d92b5a88e57c70e6bf1
Sep 16 13:16:48.640669 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.640654    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b6bvs\""
Sep 16 13:16:48.642884 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.642863    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rqgwl"]
Sep 16 13:16:48.683238 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.683215    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw9rc\" (UniqueName: \"kubernetes.io/projected/7f03aff7-7f6c-4773-9fb1-a3ce498421e4-kube-api-access-kw9rc\") pod \"authorino-59c974bd48-rqgwl\" (UID: \"7f03aff7-7f6c-4773-9fb1-a3ce498421e4\") " pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:16:48.783769 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.783745    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw9rc\" (UniqueName: \"kubernetes.io/projected/7f03aff7-7f6c-4773-9fb1-a3ce498421e4-kube-api-access-kw9rc\") pod \"authorino-59c974bd48-rqgwl\" (UID: \"7f03aff7-7f6c-4773-9fb1-a3ce498421e4\") " pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:16:48.792272 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.792252    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw9rc\" (UniqueName: \"kubernetes.io/projected/7f03aff7-7f6c-4773-9fb1-a3ce498421e4-kube-api-access-kw9rc\") pod \"authorino-59c974bd48-rqgwl\" (UID: \"7f03aff7-7f6c-4773-9fb1-a3ce498421e4\") " pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:16:48.799222 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.799199    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd" event={"ID":"d517954a-0f70-4689-bf72-edc01d46b1b1","Type":"ContainerStarted","Data":"7c69763e3bd0dc750ea8b01abf0b3e842823495cf2211d92b5a88e57c70e6bf1"}
Sep 16 13:16:48.953728 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:48.953654    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:16:49.221539 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:49.221447    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rqgwl"]
Sep 16 13:16:49.225049 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:16:49.225017    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f03aff7_7f6c_4773_9fb1_a3ce498421e4.slice/crio-7b879df9d6a5c583fc9c369c1f119286e75c4c1d7e7585d97439351d9ad6a3f4 WatchSource:0}: Error finding container 7b879df9d6a5c583fc9c369c1f119286e75c4c1d7e7585d97439351d9ad6a3f4: Status 404 returned error can't find the container with id 7b879df9d6a5c583fc9c369c1f119286e75c4c1d7e7585d97439351d9ad6a3f4
Sep 16 13:16:49.801391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:49.801358    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rqgwl" event={"ID":"7f03aff7-7f6c-4773-9fb1-a3ce498421e4","Type":"ContainerStarted","Data":"7b879df9d6a5c583fc9c369c1f119286e75c4c1d7e7585d97439351d9ad6a3f4"}
Sep 16 13:16:50.807452 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:50.807407    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd" event={"ID":"d517954a-0f70-4689-bf72-edc01d46b1b1","Type":"ContainerStarted","Data":"e947943c6da76f7f303698e518cf22c7ca2edd6c8574f885fe04d3406412d529"}
Sep 16 13:16:50.807963 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:50.807825    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:16:50.834004 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:50.833961    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd" podStartSLOduration=0.914682319 podStartE2EDuration="2.833947136s" totalImagesPullingTime="1.919264817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:16:50.832564236 +0000 UTC m=+518.509278672" watchObservedRunningTime="2026-09-16 13:16:50.833947136 +0000 UTC m=+518.510661572"
Sep 16 13:16:51.812844 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:51.812801    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rqgwl" event={"ID":"7f03aff7-7f6c-4773-9fb1-a3ce498421e4","Type":"ContainerStarted","Data":"b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb"}
Sep 16 13:16:51.832949 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:16:51.832906    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-rqgwl" podStartSLOduration=1.461142214 podStartE2EDuration="3.832891263s" totalImagesPullingTime="2.371749049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:16:51.830969265 +0000 UTC m=+519.507683697" watchObservedRunningTime="2026-09-16 13:16:51.832891263 +0000 UTC m=+519.509605699"
Sep 16 13:17:01.813699 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:01.813659    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"
Sep 16 13:17:20.219153 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.219080    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-8m4np"]
Sep 16 13:17:20.221314 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.221298    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:20.237063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.237039    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-8m4np"]
Sep 16 13:17:20.313970 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.313949    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sz79\" (UniqueName: \"kubernetes.io/projected/f37c9428-e504-4e09-b442-d39807057959-kube-api-access-8sz79\") pod \"authorino-557f876f69-8m4np\" (UID: \"f37c9428-e504-4e09-b442-d39807057959\") " pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:20.381858 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.381835    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-8m4np"]
Sep 16 13:17:20.382015 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:17:20.381998    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8sz79], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-8m4np" podUID="f37c9428-e504-4e09-b442-d39807057959"
Sep 16 13:17:20.414509 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.414484    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sz79\" (UniqueName: \"kubernetes.io/projected/f37c9428-e504-4e09-b442-d39807057959-kube-api-access-8sz79\") pod \"authorino-557f876f69-8m4np\" (UID: \"f37c9428-e504-4e09-b442-d39807057959\") " pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:20.451016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.450989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sz79\" (UniqueName: \"kubernetes.io/projected/f37c9428-e504-4e09-b442-d39807057959-kube-api-access-8sz79\") pod \"authorino-557f876f69-8m4np\" (UID: \"f37c9428-e504-4e09-b442-d39807057959\") " pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:20.928120 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.928089    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:20.934151 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:20.934129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:21.019090 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:21.019059    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sz79\" (UniqueName: \"kubernetes.io/projected/f37c9428-e504-4e09-b442-d39807057959-kube-api-access-8sz79\") pod \"f37c9428-e504-4e09-b442-d39807057959\" (UID: \"f37c9428-e504-4e09-b442-d39807057959\") "
Sep 16 13:17:21.021114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:21.021087    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37c9428-e504-4e09-b442-d39807057959-kube-api-access-8sz79" (OuterVolumeSpecName: "kube-api-access-8sz79") pod "f37c9428-e504-4e09-b442-d39807057959" (UID: "f37c9428-e504-4e09-b442-d39807057959"). InnerVolumeSpecName "kube-api-access-8sz79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:17:21.119426 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:21.119404    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sz79\" (UniqueName: \"kubernetes.io/projected/f37c9428-e504-4e09-b442-d39807057959-kube-api-access-8sz79\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:17:21.932935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:21.932902    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8m4np"
Sep 16 13:17:21.988955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:21.988926    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-8m4np"]
Sep 16 13:17:21.993946 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:21.993921    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-8m4np"]
Sep 16 13:17:22.671482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.671449    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rqgwl"]
Sep 16 13:17:22.671663 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.671633    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-rqgwl" podUID="7f03aff7-7f6c-4773-9fb1-a3ce498421e4" containerName="authorino" containerID="cri-o://b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb" gracePeriod=30
Sep 16 13:17:22.800706 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.800673    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f37c9428-e504-4e09-b442-d39807057959" path="/var/lib/kubelet/pods/f37c9428-e504-4e09-b442-d39807057959/volumes"
Sep 16 13:17:22.908422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.908394    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:17:22.938641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.938572    2556 generic.go:356] "Generic (PLEG): container finished" podID="7f03aff7-7f6c-4773-9fb1-a3ce498421e4" containerID="b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb" exitCode=0
Sep 16 13:17:22.938641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.938621    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rqgwl"
Sep 16 13:17:22.938641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.938634    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rqgwl" event={"ID":"7f03aff7-7f6c-4773-9fb1-a3ce498421e4","Type":"ContainerDied","Data":"b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb"}
Sep 16 13:17:22.939000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.938672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rqgwl" event={"ID":"7f03aff7-7f6c-4773-9fb1-a3ce498421e4","Type":"ContainerDied","Data":"7b879df9d6a5c583fc9c369c1f119286e75c4c1d7e7585d97439351d9ad6a3f4"}
Sep 16 13:17:22.939000 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.938692    2556 scope.go:117] "RemoveContainer" containerID="b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb"
Sep 16 13:17:22.946504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.946487    2556 scope.go:117] "RemoveContainer" containerID="b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb"
Sep 16 13:17:22.946769 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:17:22.946753    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb\": container with ID starting with b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb not found: ID does not exist" containerID="b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb"
Sep 16 13:17:22.946834 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:22.946775    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb"} err="failed to get container status \"b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb\": rpc error: code = NotFound desc = could not find container \"b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb\": container with ID starting with b4d3ec4f148f75c89936e555ec6463305413277513182b2ab6f150f6d51fb4eb not found: ID does not exist"
Sep 16 13:17:23.035175 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:23.035145    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw9rc\" (UniqueName: \"kubernetes.io/projected/7f03aff7-7f6c-4773-9fb1-a3ce498421e4-kube-api-access-kw9rc\") pod \"7f03aff7-7f6c-4773-9fb1-a3ce498421e4\" (UID: \"7f03aff7-7f6c-4773-9fb1-a3ce498421e4\") "
Sep 16 13:17:23.037019 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:23.036994    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f03aff7-7f6c-4773-9fb1-a3ce498421e4-kube-api-access-kw9rc" (OuterVolumeSpecName: "kube-api-access-kw9rc") pod "7f03aff7-7f6c-4773-9fb1-a3ce498421e4" (UID: "7f03aff7-7f6c-4773-9fb1-a3ce498421e4"). InnerVolumeSpecName "kube-api-access-kw9rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:17:23.136347 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:23.136303    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw9rc\" (UniqueName: \"kubernetes.io/projected/7f03aff7-7f6c-4773-9fb1-a3ce498421e4-kube-api-access-kw9rc\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:17:23.268137 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:23.268107    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rqgwl"]
Sep 16 13:17:23.271576 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:23.271555    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rqgwl"]
Sep 16 13:17:24.799826 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:17:24.799786    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f03aff7-7f6c-4773-9fb1-a3ce498421e4" path="/var/lib/kubelet/pods/7f03aff7-7f6c-4773-9fb1-a3ce498421e4/volumes"
Sep 16 13:18:06.834673 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:06.834635    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:18:12.775712 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:12.775686    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:18:12.778782 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:12.778758    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:18:29.119712 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:29.119682    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:18:37.935496 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:37.935408    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:18:48.840491 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:48.840460    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:18:55.488904 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.488866    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"]
Sep 16 13:18:55.489453 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.489280    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f03aff7-7f6c-4773-9fb1-a3ce498421e4" containerName="authorino"
Sep 16 13:18:55.489453 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.489299    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f03aff7-7f6c-4773-9fb1-a3ce498421e4" containerName="authorino"
Sep 16 13:18:55.489453 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.489390    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f03aff7-7f6c-4773-9fb1-a3ce498421e4" containerName="authorino"
Sep 16 13:18:55.491373 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.491352    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.493828 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.493804    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 13:18:55.493935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.493813    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 13:18:55.494468 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.494448    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 13:18:55.494549 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.494465    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bdc7l\""
Sep 16 13:18:55.502646 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.502626    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"]
Sep 16 13:18:55.538280 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.538255    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.538365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.538296    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.538365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.538320    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.538365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.538361    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlkp\" (UniqueName: \"kubernetes.io/projected/7767e174-64d3-48dd-a0f0-4efe5e9cd660-kube-api-access-jmlkp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.538469 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.538387    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.538469 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.538411    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7767e174-64d3-48dd-a0f0-4efe5e9cd660-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.638870 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.638844    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.638883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.638904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlkp\" (UniqueName: \"kubernetes.io/projected/7767e174-64d3-48dd-a0f0-4efe5e9cd660-kube-api-access-jmlkp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.638924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.638956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7767e174-64d3-48dd-a0f0-4efe5e9cd660-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.638993    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639337 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.639276    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639376 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.639355    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.639412 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.639386    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.641056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.641036    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7767e174-64d3-48dd-a0f0-4efe5e9cd660-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.641398 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.641379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7767e174-64d3-48dd-a0f0-4efe5e9cd660-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.646880 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.646860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlkp\" (UniqueName: \"kubernetes.io/projected/7767e174-64d3-48dd-a0f0-4efe5e9cd660-kube-api-access-jmlkp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lr2mg\" (UID: \"7767e174-64d3-48dd-a0f0-4efe5e9cd660\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:55.801317 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:55.801246    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:18:56.069052 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:56.068969    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"]
Sep 16 13:18:56.072272 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:18:56.072243    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7767e174_64d3_48dd_a0f0_4efe5e9cd660.slice/crio-cd6bcfcea60aeb365ae108d0faef19646c079522767a589e4f4e49f2c110dfc4 WatchSource:0}: Error finding container cd6bcfcea60aeb365ae108d0faef19646c079522767a589e4f4e49f2c110dfc4: Status 404 returned error can't find the container with id cd6bcfcea60aeb365ae108d0faef19646c079522767a589e4f4e49f2c110dfc4
Sep 16 13:18:56.073501 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:56.073483    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:18:56.322867 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:56.322791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg" event={"ID":"7767e174-64d3-48dd-a0f0-4efe5e9cd660","Type":"ContainerStarted","Data":"cd6bcfcea60aeb365ae108d0faef19646c079522767a589e4f4e49f2c110dfc4"}
Sep 16 13:18:56.743093 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:56.743056    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:18:57.329747 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:57.329666    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg" event={"ID":"7767e174-64d3-48dd-a0f0-4efe5e9cd660","Type":"ContainerStarted","Data":"c3aeb9302b556e2e5061645d0ce6c8373bb2fea9761f2677005cefa35cc67d80"}
Sep 16 13:18:57.350471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:18:57.350418    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg" podStartSLOduration=1.438536986 podStartE2EDuration="2.350402585s" totalImagesPullingTime="911.865599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:18:57.34984167 +0000 UTC m=+645.026556110" watchObservedRunningTime="2026-09-16 13:18:57.350402585 +0000 UTC m=+645.027117019"
Sep 16 13:19:05.801784 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:05.801750    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:19:05.802281 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:05.801835    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:19:05.818882 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:05.818855    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:19:06.383447 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:06.383414    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lr2mg"
Sep 16 13:19:07.349530 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:07.349498    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:19:13.257693 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:13.257652    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:19:31.482382 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.482343    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"]
Sep 16 13:19:31.484983 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.484960    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.487810 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.487789    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 16 13:19:31.495650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.495622    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"]
Sep 16 13:19:31.610427 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.610403    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.610530 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.610432    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.610530 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.610452    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.610530 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.610471    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kslk6\" (UniqueName: \"kubernetes.io/projected/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-kube-api-access-kslk6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.610664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.610536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.610664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.610618    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711149 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kslk6\" (UniqueName: \"kubernetes.io/projected/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-kube-api-access-kslk6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711354 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711354 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711283    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711354 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711311    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711539    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711700 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711679    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.711762 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.711711    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.713761 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.713735    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.714065 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.714042    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.718966 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.718935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kslk6\" (UniqueName: \"kubernetes.io/projected/5d117f9b-9849-4bbe-b6c6-25b0b11a9c32-kube-api-access-kslk6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp\" (UID: \"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:31.794935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:31.794883    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:32.064044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:32.063977    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"]
Sep 16 13:19:32.067312 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:19:32.067283    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d117f9b_9849_4bbe_b6c6_25b0b11a9c32.slice/crio-b3c815ddfb7875b65e37ba2e2cf594592525607a729d97169ad62d1daa242502 WatchSource:0}: Error finding container b3c815ddfb7875b65e37ba2e2cf594592525607a729d97169ad62d1daa242502: Status 404 returned error can't find the container with id b3c815ddfb7875b65e37ba2e2cf594592525607a729d97169ad62d1daa242502
Sep 16 13:19:32.484799 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:32.484766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp" event={"ID":"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32","Type":"ContainerStarted","Data":"c584a853d358892200953a78a7ce0ff04a848d8ef112ca99ec44db6643eb05dc"}
Sep 16 13:19:32.485143 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:32.484815    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp" event={"ID":"5d117f9b-9849-4bbe-b6c6-25b0b11a9c32","Type":"ContainerStarted","Data":"b3c815ddfb7875b65e37ba2e2cf594592525607a729d97169ad62d1daa242502"}
Sep 16 13:19:32.501180 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:32.501124    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp" podStartSLOduration=1.329670519 podStartE2EDuration="1.50110809s" totalImagesPullingTime="171.437571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:19:32.50037683 +0000 UTC m=+680.177091267" watchObservedRunningTime="2026-09-16 13:19:32.50110809 +0000 UTC m=+680.177822521"
Sep 16 13:19:33.852785 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:33.852753    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:19:41.795344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:41.795304    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:41.795344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:41.795353    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:41.808031 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:41.808001    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:19:42.537457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:19:42.537426    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp"
Sep 16 13:20:34.029468 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.029433    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85fb6fc7f5-ddcxv"]
Sep 16 13:20:34.032500 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.032484    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.035061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.035040    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b6bvs\""
Sep 16 13:20:34.036033 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.036016    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 13:20:34.041003 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.040983    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85fb6fc7f5-ddcxv"]
Sep 16 13:20:34.082292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.082268    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc6d7472-ce26-4447-834b-6950cb9b43bb-tls-cert\") pod \"authorino-85fb6fc7f5-ddcxv\" (UID: \"dc6d7472-ce26-4447-834b-6950cb9b43bb\") " pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.082383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.082331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8mgc\" (UniqueName: \"kubernetes.io/projected/dc6d7472-ce26-4447-834b-6950cb9b43bb-kube-api-access-s8mgc\") pod \"authorino-85fb6fc7f5-ddcxv\" (UID: \"dc6d7472-ce26-4447-834b-6950cb9b43bb\") " pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.183637 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.183613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc6d7472-ce26-4447-834b-6950cb9b43bb-tls-cert\") pod \"authorino-85fb6fc7f5-ddcxv\" (UID: \"dc6d7472-ce26-4447-834b-6950cb9b43bb\") " pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.183734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.183682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8mgc\" (UniqueName: \"kubernetes.io/projected/dc6d7472-ce26-4447-834b-6950cb9b43bb-kube-api-access-s8mgc\") pod \"authorino-85fb6fc7f5-ddcxv\" (UID: \"dc6d7472-ce26-4447-834b-6950cb9b43bb\") " pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.185938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.185914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc6d7472-ce26-4447-834b-6950cb9b43bb-tls-cert\") pod \"authorino-85fb6fc7f5-ddcxv\" (UID: \"dc6d7472-ce26-4447-834b-6950cb9b43bb\") " pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.192509 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.192486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8mgc\" (UniqueName: \"kubernetes.io/projected/dc6d7472-ce26-4447-834b-6950cb9b43bb-kube-api-access-s8mgc\") pod \"authorino-85fb6fc7f5-ddcxv\" (UID: \"dc6d7472-ce26-4447-834b-6950cb9b43bb\") " pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.341777 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.341702    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv"
Sep 16 13:20:34.612042 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.611569    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85fb6fc7f5-ddcxv"]
Sep 16 13:20:34.616935 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:20:34.616893    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc6d7472_ce26_4447_834b_6950cb9b43bb.slice/crio-6b788e2c57dc4cda7c18de9dfbfba8b3da0d601301687a1eee6dfaa69a9fab59 WatchSource:0}: Error finding container 6b788e2c57dc4cda7c18de9dfbfba8b3da0d601301687a1eee6dfaa69a9fab59: Status 404 returned error can't find the container with id 6b788e2c57dc4cda7c18de9dfbfba8b3da0d601301687a1eee6dfaa69a9fab59
Sep 16 13:20:34.744260 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:34.744230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv" event={"ID":"dc6d7472-ce26-4447-834b-6950cb9b43bb","Type":"ContainerStarted","Data":"6b788e2c57dc4cda7c18de9dfbfba8b3da0d601301687a1eee6dfaa69a9fab59"}
Sep 16 13:20:35.750589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:35.750552    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv" event={"ID":"dc6d7472-ce26-4447-834b-6950cb9b43bb","Type":"ContainerStarted","Data":"48ac975fee974930f60029216aea0e90d3edbf3372176961ae6846a6146e1160"}
Sep 16 13:20:35.769711 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:20:35.769657    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85fb6fc7f5-ddcxv" podStartSLOduration=1.034438954 podStartE2EDuration="1.769639221s" totalImagesPullingTime="735.200267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:20:35.768231382 +0000 UTC m=+743.444945820" watchObservedRunningTime="2026-09-16 13:20:35.769639221 +0000 UTC m=+743.446353660"
Sep 16 13:21:01.139058 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.139024    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-1e2eb0"]
Sep 16 13:21:01.142400 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.142383    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:01.144592 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.144565    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mxtqf\""
Sep 16 13:21:01.148253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.148224    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-1e2eb0"]
Sep 16 13:21:01.296225 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.296189    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfghh\" (UniqueName: \"kubernetes.io/projected/3324783e-4af9-430f-95af-75312d287d40-kube-api-access-bfghh\") pod \"test-curl-7491-1e2eb0\" (UID: \"3324783e-4af9-430f-95af-75312d287d40\") " pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:01.397665 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.397560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfghh\" (UniqueName: \"kubernetes.io/projected/3324783e-4af9-430f-95af-75312d287d40-kube-api-access-bfghh\") pod \"test-curl-7491-1e2eb0\" (UID: \"3324783e-4af9-430f-95af-75312d287d40\") " pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:01.412689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.412661    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfghh\" (UniqueName: \"kubernetes.io/projected/3324783e-4af9-430f-95af-75312d287d40-kube-api-access-bfghh\") pod \"test-curl-7491-1e2eb0\" (UID: \"3324783e-4af9-430f-95af-75312d287d40\") " pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:01.452107 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.452074    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:01.720013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.719940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-1e2eb0"]
Sep 16 13:21:01.723698 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:01.723671    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3324783e_4af9_430f_95af_75312d287d40.slice/crio-8ba07297e2a7dedcd57af515792683400a24e49ffc527d6e9ee3beaea636578e WatchSource:0}: Error finding container 8ba07297e2a7dedcd57af515792683400a24e49ffc527d6e9ee3beaea636578e: Status 404 returned error can't find the container with id 8ba07297e2a7dedcd57af515792683400a24e49ffc527d6e9ee3beaea636578e
Sep 16 13:21:01.857924 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:01.857896    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-1e2eb0" event={"ID":"3324783e-4af9-430f-95af-75312d287d40","Type":"ContainerStarted","Data":"8ba07297e2a7dedcd57af515792683400a24e49ffc527d6e9ee3beaea636578e"}
Sep 16 13:21:02.747139 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:02.747095    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:21:03.868780 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:03.868757    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-1e2eb0" event={"ID":"3324783e-4af9-430f-95af-75312d287d40","Type":"ContainerStarted","Data":"ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777"}
Sep 16 13:21:03.883806 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:03.883762    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7491-1e2eb0" podStartSLOduration=0.817318218 podStartE2EDuration="2.883746902s" totalImagesPullingTime="2.066428684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:03.882989159 +0000 UTC m=+771.559703595" watchObservedRunningTime="2026-09-16 13:21:03.883746902 +0000 UTC m=+771.560461336"
Sep 16 13:21:04.511094 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.511056    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-1e2eb0"]
Sep 16 13:21:04.518189 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.518160    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-1e2eb0"]
Sep 16 13:21:04.656220 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.656192    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-2d8137"]
Sep 16 13:21:04.656510 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.656496    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3324783e-4af9-430f-95af-75312d287d40" containerName="test-curl-7491-1e2eb0"
Sep 16 13:21:04.656563 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.656512    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3324783e-4af9-430f-95af-75312d287d40" containerName="test-curl-7491-1e2eb0"
Sep 16 13:21:04.656610 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.656582    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3324783e-4af9-430f-95af-75312d287d40" containerName="test-curl-7491-1e2eb0"
Sep 16 13:21:04.659659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.659637    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:04.664715 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.664668    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-2d8137"]
Sep 16 13:21:04.725436 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.725404    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6srt\" (UniqueName: \"kubernetes.io/projected/fb5b5fa7-eec5-4522-a879-34a425c09bcd-kube-api-access-w6srt\") pod \"test-curl-7491-2d8137\" (UID: \"fb5b5fa7-eec5-4522-a879-34a425c09bcd\") " pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:04.826686 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.826625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6srt\" (UniqueName: \"kubernetes.io/projected/fb5b5fa7-eec5-4522-a879-34a425c09bcd-kube-api-access-w6srt\") pod \"test-curl-7491-2d8137\" (UID: \"fb5b5fa7-eec5-4522-a879-34a425c09bcd\") " pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:04.835165 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.835138    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6srt\" (UniqueName: \"kubernetes.io/projected/fb5b5fa7-eec5-4522-a879-34a425c09bcd-kube-api-access-w6srt\") pod \"test-curl-7491-2d8137\" (UID: \"fb5b5fa7-eec5-4522-a879-34a425c09bcd\") " pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:04.874488 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:04.874457    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7491-1e2eb0" container="test-curl-7491-1e2eb0"
Sep 16 13:21:04.972020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:04.971986    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:05.244494 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:05.244462    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-2d8137"]
Sep 16 13:21:05.247694 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:05.247661    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb5b5fa7_eec5_4522_a879_34a425c09bcd.slice/crio-8557e5fd5542995f04d71c0f8eb5dbcec14ca17f0a4dcae99a466b0a6a560c09 WatchSource:0}: Error finding container 8557e5fd5542995f04d71c0f8eb5dbcec14ca17f0a4dcae99a466b0a6a560c09: Status 404 returned error can't find the container with id 8557e5fd5542995f04d71c0f8eb5dbcec14ca17f0a4dcae99a466b0a6a560c09
Sep 16 13:21:05.879700 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:05.879654    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-2d8137" event={"ID":"fb5b5fa7-eec5-4522-a879-34a425c09bcd","Type":"ContainerStarted","Data":"190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7"}
Sep 16 13:21:05.880316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:05.879713    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-2d8137" event={"ID":"fb5b5fa7-eec5-4522-a879-34a425c09bcd","Type":"ContainerStarted","Data":"8557e5fd5542995f04d71c0f8eb5dbcec14ca17f0a4dcae99a466b0a6a560c09"}
Sep 16 13:21:05.880316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:05.879802    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7491-1e2eb0" podUID="3324783e-4af9-430f-95af-75312d287d40" containerName="test-curl-7491-1e2eb0" containerID="cri-o://ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777" gracePeriod=2
Sep 16 13:21:06.552442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:06.552388    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7491-2d8137" podStartSLOduration=2.5523698870000002 podStartE2EDuration="2.552369887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:05.895416907 +0000 UTC m=+773.572131342" watchObservedRunningTime="2026-09-16 13:21:06.552369887 +0000 UTC m=+774.229084321"
Sep 16 13:21:06.553256 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:06.553238    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-2d8137"]
Sep 16 13:21:06.559438 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:06.559412    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-2d8137"]
Sep 16 13:21:06.883241 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:06.883210    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7491-2d8137" container="test-curl-7491-2d8137"
Sep 16 13:21:07.722183 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.722151    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"]
Sep 16 13:21:07.722459 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.722447    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5b5fa7-eec5-4522-a879-34a425c09bcd" containerName="test-curl-7491-2d8137"
Sep 16 13:21:07.722523 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.722461    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5b5fa7-eec5-4522-a879-34a425c09bcd" containerName="test-curl-7491-2d8137"
Sep 16 13:21:07.722523 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.722512    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5b5fa7-eec5-4522-a879-34a425c09bcd" containerName="test-curl-7491-2d8137"
Sep 16 13:21:07.725689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.725672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.727760 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.727740    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-a21a40\""
Sep 16 13:21:07.727894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.727877    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-a21a40-dockercfg-fbnqs\""
Sep 16 13:21:07.733692 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.733672    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"]
Sep 16 13:21:07.750426 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.750403    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb26fad-12f8-49f4-8d6e-2e52d3cc8139-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49\" (UID: \"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.750519 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.750437    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqqh7\" (UniqueName: \"kubernetes.io/projected/ecb26fad-12f8-49f4-8d6e-2e52d3cc8139-kube-api-access-gqqh7\") pod \"payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49\" (UID: \"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.851584 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.851555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb26fad-12f8-49f4-8d6e-2e52d3cc8139-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49\" (UID: \"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.851695 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.851632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqqh7\" (UniqueName: \"kubernetes.io/projected/ecb26fad-12f8-49f4-8d6e-2e52d3cc8139-kube-api-access-gqqh7\") pod \"payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49\" (UID: \"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.852271 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.852251    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb26fad-12f8-49f4-8d6e-2e52d3cc8139-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49\" (UID: \"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.864693 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.864674    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqqh7\" (UniqueName: \"kubernetes.io/projected/ecb26fad-12f8-49f4-8d6e-2e52d3cc8139-kube-api-access-gqqh7\") pod \"payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49\" (UID: \"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:07.887485 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:07.887441    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7491-2d8137" podUID="fb5b5fa7-eec5-4522-a879-34a425c09bcd" containerName="test-curl-7491-2d8137" containerID="cri-o://190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7" gracePeriod=2
Sep 16 13:21:08.037084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.037055    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:08.058314 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.058293    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:08.154217 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.154184    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfghh\" (UniqueName: \"kubernetes.io/projected/3324783e-4af9-430f-95af-75312d287d40-kube-api-access-bfghh\") pod \"3324783e-4af9-430f-95af-75312d287d40\" (UID: \"3324783e-4af9-430f-95af-75312d287d40\") "
Sep 16 13:21:08.156222 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.156193    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3324783e-4af9-430f-95af-75312d287d40-kube-api-access-bfghh" (OuterVolumeSpecName: "kube-api-access-bfghh") pod "3324783e-4af9-430f-95af-75312d287d40" (UID: "3324783e-4af9-430f-95af-75312d287d40"). InnerVolumeSpecName "kube-api-access-bfghh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:21:08.255005 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.254941    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfghh\" (UniqueName: \"kubernetes.io/projected/3324783e-4af9-430f-95af-75312d287d40-kube-api-access-bfghh\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:08.309452 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.309368    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"]
Sep 16 13:21:08.312205 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:08.312181    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb26fad_12f8_49f4_8d6e_2e52d3cc8139.slice/crio-3bd3dce23b280c7de6d43bb34c84ff96162bf7e5cf24cbe883a01549caf7490a WatchSource:0}: Error finding container 3bd3dce23b280c7de6d43bb34c84ff96162bf7e5cf24cbe883a01549caf7490a: Status 404 returned error can't find the container with id 3bd3dce23b280c7de6d43bb34c84ff96162bf7e5cf24cbe883a01549caf7490a
Sep 16 13:21:08.799823 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.799792    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3324783e-4af9-430f-95af-75312d287d40" path="/var/lib/kubelet/pods/3324783e-4af9-430f-95af-75312d287d40/volumes"
Sep 16 13:21:08.892800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.892773    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49" event={"ID":"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139","Type":"ContainerStarted","Data":"3bd3dce23b280c7de6d43bb34c84ff96162bf7e5cf24cbe883a01549caf7490a"}
Sep 16 13:21:08.894008 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.893986    2556 generic.go:356] "Generic (PLEG): container finished" podID="3324783e-4af9-430f-95af-75312d287d40" containerID="ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777" exitCode=137
Sep 16 13:21:08.894095 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.894028    2556 scope.go:117] "RemoveContainer" containerID="ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777"
Sep 16 13:21:08.894137 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.894031    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-1e2eb0"
Sep 16 13:21:08.901715 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.901698    2556 scope.go:117] "RemoveContainer" containerID="ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777"
Sep 16 13:21:08.902008 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:08.901972    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777\": container with ID starting with ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777 not found: ID does not exist" containerID="ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777"
Sep 16 13:21:08.902085 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:08.902019    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777"} err="failed to get container status \"ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777\": rpc error: code = NotFound desc = could not find container \"ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777\": container with ID starting with ef705d431027761314ab3c2c604344e07eb4bc445a9a0dc1e8800419ea9f6777 not found: ID does not exist"
Sep 16 13:21:09.900925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:09.900874    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49" event={"ID":"ecb26fad-12f8-49f4-8d6e-2e52d3cc8139","Type":"ContainerStarted","Data":"8baa1ee9cbf5d5d3a59c856af36a0ba6c5cd8f505138db3a62b6ce1ba0adc4cf"}
Sep 16 13:21:09.901235 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:09.900973    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:09.923796 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:09.923751    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49" podStartSLOduration=1.537129406 podStartE2EDuration="2.923735817s" totalImagesPullingTime="1.386606411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:09.922166369 +0000 UTC m=+777.598880803" watchObservedRunningTime="2026-09-16 13:21:09.923735817 +0000 UTC m=+777.600450251"
Sep 16 13:21:10.085160 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.085130    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:10.172492 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.172453    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6srt\" (UniqueName: \"kubernetes.io/projected/fb5b5fa7-eec5-4522-a879-34a425c09bcd-kube-api-access-w6srt\") pod \"fb5b5fa7-eec5-4522-a879-34a425c09bcd\" (UID: \"fb5b5fa7-eec5-4522-a879-34a425c09bcd\") "
Sep 16 13:21:10.174728 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.174701    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5b5fa7-eec5-4522-a879-34a425c09bcd-kube-api-access-w6srt" (OuterVolumeSpecName: "kube-api-access-w6srt") pod "fb5b5fa7-eec5-4522-a879-34a425c09bcd" (UID: "fb5b5fa7-eec5-4522-a879-34a425c09bcd"). InnerVolumeSpecName "kube-api-access-w6srt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:21:10.273616 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.273528    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6srt\" (UniqueName: \"kubernetes.io/projected/fb5b5fa7-eec5-4522-a879-34a425c09bcd-kube-api-access-w6srt\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:10.800380 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.800346    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5b5fa7-eec5-4522-a879-34a425c09bcd" path="/var/lib/kubelet/pods/fb5b5fa7-eec5-4522-a879-34a425c09bcd/volumes"
Sep 16 13:21:10.909573 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.909535    2556 generic.go:356] "Generic (PLEG): container finished" podID="fb5b5fa7-eec5-4522-a879-34a425c09bcd" containerID="190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7" exitCode=137
Sep 16 13:21:10.910289 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.910256    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-2d8137"
Sep 16 13:21:10.910589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.910571    2556 scope.go:117] "RemoveContainer" containerID="190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7"
Sep 16 13:21:10.922370 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.922343    2556 scope.go:117] "RemoveContainer" containerID="190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7"
Sep 16 13:21:10.922765 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:10.922738    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7\": container with ID starting with 190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7 not found: ID does not exist" containerID="190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7"
Sep 16 13:21:10.922930 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:10.922895    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7"} err="failed to get container status \"190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7\": rpc error: code = NotFound desc = could not find container \"190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7\": container with ID starting with 190d31caf393445aabd3837b853bf20c939314c4896dc09d07e96ed65e9493a7 not found: ID does not exist"
Sep 16 13:21:14.897913 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.897882    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"]
Sep 16 13:21:14.902390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.902374    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:14.905305 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.905280    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ef119741-serving-cert\""
Sep 16 13:21:14.905305 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.905298    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 13:21:14.905305 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.905310    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:21:14.905512 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.905307    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tmb4l\""
Sep 16 13:21:14.905512 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.905362    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 13:21:14.908436 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:14.908415    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"]
Sep 16 13:21:15.013001 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.012967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78cc808-c757-4c63-a497-8124aa121109-openshift-service-ca\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.013119 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.013055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.013119 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.013083    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5j5\" (UniqueName: \"kubernetes.io/projected/e78cc808-c757-4c63-a497-8124aa121109-kube-api-access-hm5j5\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.013211 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.013152    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-metrics-certs\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.114219 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.114192    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.114316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.114222    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5j5\" (UniqueName: \"kubernetes.io/projected/e78cc808-c757-4c63-a497-8124aa121109-kube-api-access-hm5j5\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.114316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.114252    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-metrics-certs\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.114316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.114297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78cc808-c757-4c63-a497-8124aa121109-openshift-service-ca\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.114438 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:15.114338    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-serving-cert: secret "maas-api-e2e-ait-ef119741-serving-cert" not found
Sep 16 13:21:15.114438 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:15.114412    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls podName:e78cc808-c757-4c63-a497-8124aa121109 nodeName:}" failed. No retries permitted until 2026-09-16 13:21:15.614390593 +0000 UTC m=+783.291105024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls") pod "maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" (UID: "e78cc808-c757-4c63-a497-8124aa121109") : secret "maas-api-e2e-ait-ef119741-serving-cert" not found
Sep 16 13:21:15.114939 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.114918    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78cc808-c757-4c63-a497-8124aa121109-openshift-service-ca\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.116794 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.116773    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-metrics-certs\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.126631 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.126587    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5j5\" (UniqueName: \"kubernetes.io/projected/e78cc808-c757-4c63-a497-8124aa121109-kube-api-access-hm5j5\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.618284 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.618252    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.620548 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.620527    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls\") pod \"maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.812799 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.812772    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:15.876261 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.876185    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"]
Sep 16 13:21:15.881264 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.881246    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:15.883344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.883324    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ef119741\""
Sep 16 13:21:15.892514 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.892493    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"]
Sep 16 13:21:15.921218 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.921195    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f119a55-ef58-4aa4-b521-71cc7dc46783-plugins-config-volume\") pod \"payload-processing-e2e-ait-ef119741-58f684c-m9947\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") " pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:15.921508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.921241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnfpx\" (UniqueName: \"kubernetes.io/projected/4f119a55-ef58-4aa4-b521-71cc7dc46783-kube-api-access-jnfpx\") pod \"payload-processing-e2e-ait-ef119741-58f684c-m9947\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") " pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:15.945828 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.945795    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"]
Sep 16 13:21:15.949144 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.949125    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:15.951056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.951037    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ef119741-dockercfg-w686b\""
Sep 16 13:21:15.956938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:15.956915    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"]
Sep 16 13:21:16.022208 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.022183    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f119a55-ef58-4aa4-b521-71cc7dc46783-plugins-config-volume\") pod \"payload-processing-e2e-ait-ef119741-58f684c-m9947\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") " pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:16.022344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.022217    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d515a04-bd9a-463c-9bdd-528ef19fad04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.022344 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.022251    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnfpx\" (UniqueName: \"kubernetes.io/projected/4f119a55-ef58-4aa4-b521-71cc7dc46783-kube-api-access-jnfpx\") pod \"payload-processing-e2e-ait-ef119741-58f684c-m9947\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") " pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:16.022447 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.022339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5nl9\" (UniqueName: \"kubernetes.io/projected/7d515a04-bd9a-463c-9bdd-528ef19fad04-kube-api-access-m5nl9\") pod \"payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.022889 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.022873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f119a55-ef58-4aa4-b521-71cc7dc46783-plugins-config-volume\") pod \"payload-processing-e2e-ait-ef119741-58f684c-m9947\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") " pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:16.029961 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.029945    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnfpx\" (UniqueName: \"kubernetes.io/projected/4f119a55-ef58-4aa4-b521-71cc7dc46783-kube-api-access-jnfpx\") pod \"payload-processing-e2e-ait-ef119741-58f684c-m9947\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") " pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:16.085127 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.085014    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"]
Sep 16 13:21:16.087911 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:16.087886    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78cc808_c757_4c63_a497_8124aa121109.slice/crio-33fbba66c14a0572a82a7df1202b2e6c420abdd78ffa3eb19ebf57bcbdc23c96 WatchSource:0}: Error finding container 33fbba66c14a0572a82a7df1202b2e6c420abdd78ffa3eb19ebf57bcbdc23c96: Status 404 returned error can't find the container with id 33fbba66c14a0572a82a7df1202b2e6c420abdd78ffa3eb19ebf57bcbdc23c96
Sep 16 13:21:16.123203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.123174    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d515a04-bd9a-463c-9bdd-528ef19fad04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.123319 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.123284    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5nl9\" (UniqueName: \"kubernetes.io/projected/7d515a04-bd9a-463c-9bdd-528ef19fad04-kube-api-access-m5nl9\") pod \"payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.124007 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.123989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d515a04-bd9a-463c-9bdd-528ef19fad04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.130782 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.130733    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5nl9\" (UniqueName: \"kubernetes.io/projected/7d515a04-bd9a-463c-9bdd-528ef19fad04-kube-api-access-m5nl9\") pod \"payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.194732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.194709    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:16.259521 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.259494    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.470547 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.470515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"]
Sep 16 13:21:16.473094 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:16.473071    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f119a55_ef58_4aa4_b521_71cc7dc46783.slice/crio-ac09af6057471de2563eec21c8750607076c5621f066dcfd84fcf9c45a394f79 WatchSource:0}: Error finding container ac09af6057471de2563eec21c8750607076c5621f066dcfd84fcf9c45a394f79: Status 404 returned error can't find the container with id ac09af6057471de2563eec21c8750607076c5621f066dcfd84fcf9c45a394f79
Sep 16 13:21:16.531056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.531033    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"]
Sep 16 13:21:16.532571 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:16.532547    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d515a04_bd9a_463c_9bdd_528ef19fad04.slice/crio-5fb3b55610d27cfa0528b65ec6c1dda3d89c20aaa157d78e4030aced48706acb WatchSource:0}: Error finding container 5fb3b55610d27cfa0528b65ec6c1dda3d89c20aaa157d78e4030aced48706acb: Status 404 returned error can't find the container with id 5fb3b55610d27cfa0528b65ec6c1dda3d89c20aaa157d78e4030aced48706acb
Sep 16 13:21:16.940729 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.940683    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx" event={"ID":"7d515a04-bd9a-463c-9bdd-528ef19fad04","Type":"ContainerStarted","Data":"0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb"}
Sep 16 13:21:16.941266 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.940749    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx" event={"ID":"7d515a04-bd9a-463c-9bdd-528ef19fad04","Type":"ContainerStarted","Data":"5fb3b55610d27cfa0528b65ec6c1dda3d89c20aaa157d78e4030aced48706acb"}
Sep 16 13:21:16.941266 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.940789    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:16.942800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.942767    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947" event={"ID":"4f119a55-ef58-4aa4-b521-71cc7dc46783","Type":"ContainerStarted","Data":"e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f"}
Sep 16 13:21:16.942980 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.942823    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947" event={"ID":"4f119a55-ef58-4aa4-b521-71cc7dc46783","Type":"ContainerStarted","Data":"ac09af6057471de2563eec21c8750607076c5621f066dcfd84fcf9c45a394f79"}
Sep 16 13:21:16.942980 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.942855    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:16.944185 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.944156    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" event={"ID":"e78cc808-c757-4c63-a497-8124aa121109","Type":"ContainerStarted","Data":"33fbba66c14a0572a82a7df1202b2e6c420abdd78ffa3eb19ebf57bcbdc23c96"}
Sep 16 13:21:16.957726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.957674    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx" podStartSLOduration=1.957653562 podStartE2EDuration="1.957653562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:16.955804595 +0000 UTC m=+784.632519033" watchObservedRunningTime="2026-09-16 13:21:16.957653562 +0000 UTC m=+784.634367999"
Sep 16 13:21:16.972391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:16.972345    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947" podStartSLOduration=1.972329946 podStartE2EDuration="1.972329946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:16.97149736 +0000 UTC m=+784.648211797" watchObservedRunningTime="2026-09-16 13:21:16.972329946 +0000 UTC m=+784.649044384"
Sep 16 13:21:18.956904 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:18.956868    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" event={"ID":"e78cc808-c757-4c63-a497-8124aa121109","Type":"ContainerStarted","Data":"83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758"}
Sep 16 13:21:18.957263 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:18.956959    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:18.979127 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:18.979084    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" podStartSLOduration=2.541893747 podStartE2EDuration="4.979070449s" totalImagesPullingTime="2.437176702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:18.977425699 +0000 UTC m=+786.654140133" watchObservedRunningTime="2026-09-16 13:21:18.979070449 +0000 UTC m=+786.655784883"
Sep 16 13:21:20.912579 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:20.912547    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49"
Sep 16 13:21:23.069986 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:23.069944    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:21:24.344140 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.344104    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"]
Sep 16 13:21:24.348389 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.348367    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.350427 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.350406    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c71ad7-serving-cert\""
Sep 16 13:21:24.360171 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.360144    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"]
Sep 16 13:21:24.398577 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.398547    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.398719 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.398581    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-metrics-certs\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.398719 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.398659    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfad9548-8033-44fc-892f-e4ec78c9a61d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.398824 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.398736    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pprm\" (UniqueName: \"kubernetes.io/projected/cfad9548-8033-44fc-892f-e4ec78c9a61d-kube-api-access-6pprm\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.500140 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.500108    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-metrics-certs\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.500292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.500163    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfad9548-8033-44fc-892f-e4ec78c9a61d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.500292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.500212    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pprm\" (UniqueName: \"kubernetes.io/projected/cfad9548-8033-44fc-892f-e4ec78c9a61d-kube-api-access-6pprm\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.500292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.500247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.500433 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:24.500322    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-serving-cert: secret "maas-api-e2e-inf-b-c71ad7-serving-cert" not found
Sep 16 13:21:24.500433 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:24.500374    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-maas-api-tls podName:cfad9548-8033-44fc-892f-e4ec78c9a61d nodeName:}" failed. No retries permitted until 2026-09-16 13:21:25.000359562 +0000 UTC m=+792.677073976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-maas-api-tls") pod "maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8" (UID: "cfad9548-8033-44fc-892f-e4ec78c9a61d") : secret "maas-api-e2e-inf-b-c71ad7-serving-cert" not found
Sep 16 13:21:24.500911 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.500886    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfad9548-8033-44fc-892f-e4ec78c9a61d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.502508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.502488    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-metrics-certs\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.512363 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.512335    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pprm\" (UniqueName: \"kubernetes.io/projected/cfad9548-8033-44fc-892f-e4ec78c9a61d-kube-api-access-6pprm\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:24.967420 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:24.967387    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:25.006272 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.006234    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:25.008559 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.008532    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfad9548-8033-44fc-892f-e4ec78c9a61d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8\" (UID: \"cfad9548-8033-44fc-892f-e4ec78c9a61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:25.261341 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.261251    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:25.432295 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.432261    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"]
Sep 16 13:21:25.436500 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.436482    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.438853 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.438833    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c71ad7\""
Sep 16 13:21:25.438938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.438902    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c71ad7-dockercfg-4qtwx\""
Sep 16 13:21:25.449909 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.449887    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"]
Sep 16 13:21:25.498276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.498252    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"]
Sep 16 13:21:25.501425 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.501411    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.509927 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.509904    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2aea88a-9474-43bc-a8df-d0fe082cf3a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff\" (UID: \"f2aea88a-9474-43bc-a8df-d0fe082cf3a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.510168 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.510148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vtmr\" (UniqueName: \"kubernetes.io/projected/f2aea88a-9474-43bc-a8df-d0fe082cf3a8-kube-api-access-5vtmr\") pod \"payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff\" (UID: \"f2aea88a-9474-43bc-a8df-d0fe082cf3a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.511638 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.511578    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"]
Sep 16 13:21:25.538083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.538062    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"]
Sep 16 13:21:25.539857 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:25.539834    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfad9548_8033_44fc_892f_e4ec78c9a61d.slice/crio-e9e98e1077fe6fa8f7ba001306808f210b7994a351f4bba71a650f7fa2ff271d WatchSource:0}: Error finding container e9e98e1077fe6fa8f7ba001306808f210b7994a351f4bba71a650f7fa2ff271d: Status 404 returned error can't find the container with id e9e98e1077fe6fa8f7ba001306808f210b7994a351f4bba71a650f7fa2ff271d
Sep 16 13:21:25.611540 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.611501    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdzfr\" (UniqueName: \"kubernetes.io/projected/6e767317-491f-4b74-b6ec-64e488ea4c0a-kube-api-access-bdzfr\") pod \"payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9\" (UID: \"6e767317-491f-4b74-b6ec-64e488ea4c0a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.611709 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.611626    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2aea88a-9474-43bc-a8df-d0fe082cf3a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff\" (UID: \"f2aea88a-9474-43bc-a8df-d0fe082cf3a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.611771 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.611697    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e767317-491f-4b74-b6ec-64e488ea4c0a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9\" (UID: \"6e767317-491f-4b74-b6ec-64e488ea4c0a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.611771 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.611760    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vtmr\" (UniqueName: \"kubernetes.io/projected/f2aea88a-9474-43bc-a8df-d0fe082cf3a8-kube-api-access-5vtmr\") pod \"payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff\" (UID: \"f2aea88a-9474-43bc-a8df-d0fe082cf3a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.612382 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.612357    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2aea88a-9474-43bc-a8df-d0fe082cf3a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff\" (UID: \"f2aea88a-9474-43bc-a8df-d0fe082cf3a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.619151 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.619130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vtmr\" (UniqueName: \"kubernetes.io/projected/f2aea88a-9474-43bc-a8df-d0fe082cf3a8-kube-api-access-5vtmr\") pod \"payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff\" (UID: \"f2aea88a-9474-43bc-a8df-d0fe082cf3a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.712576 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.712553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzfr\" (UniqueName: \"kubernetes.io/projected/6e767317-491f-4b74-b6ec-64e488ea4c0a-kube-api-access-bdzfr\") pod \"payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9\" (UID: \"6e767317-491f-4b74-b6ec-64e488ea4c0a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.712675 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.712652    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e767317-491f-4b74-b6ec-64e488ea4c0a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9\" (UID: \"6e767317-491f-4b74-b6ec-64e488ea4c0a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.713234 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.713218    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e767317-491f-4b74-b6ec-64e488ea4c0a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9\" (UID: \"6e767317-491f-4b74-b6ec-64e488ea4c0a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.727173 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.727151    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdzfr\" (UniqueName: \"kubernetes.io/projected/6e767317-491f-4b74-b6ec-64e488ea4c0a-kube-api-access-bdzfr\") pod \"payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9\" (UID: \"6e767317-491f-4b74-b6ec-64e488ea4c0a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.747032 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.747013    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:25.812380 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.812353    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:25.993492 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.993462    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8" event={"ID":"cfad9548-8033-44fc-892f-e4ec78c9a61d","Type":"ContainerStarted","Data":"696f51fd72d17c73dcb7f985b0659cd563cecf64a5b6586cf6ffa5aed12eb68d"}
Sep 16 13:21:25.993687 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.993522    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8" event={"ID":"cfad9548-8033-44fc-892f-e4ec78c9a61d","Type":"ContainerStarted","Data":"e9e98e1077fe6fa8f7ba001306808f210b7994a351f4bba71a650f7fa2ff271d"}
Sep 16 13:21:25.993687 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:25.993566    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:26.012036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:26.011992    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8" podStartSLOduration=1.6584403829999999 podStartE2EDuration="2.011977457s" totalImagesPullingTime="353.537074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:26.011013572 +0000 UTC m=+793.687728006" watchObservedRunningTime="2026-09-16 13:21:26.011977457 +0000 UTC m=+793.688691892"
Sep 16 13:21:26.017037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:26.016936    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"]
Sep 16 13:21:26.019332 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:26.019307    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2aea88a_9474_43bc_a8df_d0fe082cf3a8.slice/crio-2d600bab2c0fabab201836abb405cc2dc8a4621ac12b874a2807201f672b9f28 WatchSource:0}: Error finding container 2d600bab2c0fabab201836abb405cc2dc8a4621ac12b874a2807201f672b9f28: Status 404 returned error can't find the container with id 2d600bab2c0fabab201836abb405cc2dc8a4621ac12b874a2807201f672b9f28
Sep 16 13:21:26.084046 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:26.083917    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"]
Sep 16 13:21:26.085734 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:26.085710    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e767317_491f_4b74_b6ec_64e488ea4c0a.slice/crio-277305dc3071a0d19e19008d3e5bca80446e83038509358277accf8bb58a339c WatchSource:0}: Error finding container 277305dc3071a0d19e19008d3e5bca80446e83038509358277accf8bb58a339c: Status 404 returned error can't find the container with id 277305dc3071a0d19e19008d3e5bca80446e83038509358277accf8bb58a339c
Sep 16 13:21:26.999853 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:26.999814    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff" event={"ID":"f2aea88a-9474-43bc-a8df-d0fe082cf3a8","Type":"ContainerStarted","Data":"0a565f5b2c335a49cdda7be997a8e88062224a6be376543562cb0e60f3a9a905"}
Sep 16 13:21:27.000273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:26.999864    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff" event={"ID":"f2aea88a-9474-43bc-a8df-d0fe082cf3a8","Type":"ContainerStarted","Data":"2d600bab2c0fabab201836abb405cc2dc8a4621ac12b874a2807201f672b9f28"}
Sep 16 13:21:27.000273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:26.999904    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:27.001391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:27.001366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9" event={"ID":"6e767317-491f-4b74-b6ec-64e488ea4c0a","Type":"ContainerStarted","Data":"64be51d2fd3dda372327a1f7229eea2882701d0ee2d929705606329eaa96551a"}
Sep 16 13:21:27.001509 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:27.001415    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9" event={"ID":"6e767317-491f-4b74-b6ec-64e488ea4c0a","Type":"ContainerStarted","Data":"277305dc3071a0d19e19008d3e5bca80446e83038509358277accf8bb58a339c"}
Sep 16 13:21:27.020471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:27.020429    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff" podStartSLOduration=2.020415929 podStartE2EDuration="2.020415929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:27.019398529 +0000 UTC m=+794.696112964" watchObservedRunningTime="2026-09-16 13:21:27.020415929 +0000 UTC m=+794.697130364"
Sep 16 13:21:27.035623 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:27.035568    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9" podStartSLOduration=2.035557063 podStartE2EDuration="2.035557063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:27.033864162 +0000 UTC m=+794.710578597" watchObservedRunningTime="2026-09-16 13:21:27.035557063 +0000 UTC m=+794.712271497"
Sep 16 13:21:27.952331 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:27.952302    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:27.952511 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:27.952373    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:28.006869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:28.006835    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:32.006362 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:32.006334    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8"
Sep 16 13:21:35.477153 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.477072    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"]
Sep 16 13:21:35.481885 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.481863    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.485131 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.485108    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f27ea8\"/\"openshift-service-ca.crt\""
Sep 16 13:21:35.485247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.485152    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f27ea8\"/\"kube-root-ca.crt\""
Sep 16 13:21:35.485247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.485117    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f27ea8\"/\"default-dockercfg-2ckdl\""
Sep 16 13:21:35.485336 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.485243    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f27ea8\"/\"simulator-w4-f27ea8-kserve-self-signed-certs\""
Sep 16 13:21:35.490831 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.490809    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"]
Sep 16 13:21:35.589533 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.589505    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tmp-dir\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.589708 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.589551    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-dshm\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.589708 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.589572    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-home\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.589708 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.589614    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-model-cache\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.589708 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.589650    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctg5l\" (UniqueName: \"kubernetes.io/projected/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-kube-api-access-ctg5l\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.589873 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.589738    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tls-certs\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctg5l\" (UniqueName: \"kubernetes.io/projected/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-kube-api-access-ctg5l\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690375    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tls-certs\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690405    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tmp-dir\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-dshm\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690452    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-home\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-model-cache\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690841 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690817    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tmp-dir\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-model-cache\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.690956 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.690876    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-home\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.692727 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.692703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-dshm\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.692911 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.692897    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tls-certs\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.698941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.698923    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctg5l\" (UniqueName: \"kubernetes.io/projected/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-kube-api-access-ctg5l\") pod \"simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") " pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:35.792167 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:35.792114    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:36.092983 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:36.092903    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"]
Sep 16 13:21:36.097529 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:21:36.097502    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb58f4381_54b8_4ee0_a185_1bec0ef2c2a5.slice/crio-387fb2adf24b0e17b253a66b70672a6133a4e3a615db2723b551ab87e906e1ce WatchSource:0}: Error finding container 387fb2adf24b0e17b253a66b70672a6133a4e3a615db2723b551ab87e906e1ce: Status 404 returned error can't find the container with id 387fb2adf24b0e17b253a66b70672a6133a4e3a615db2723b551ab87e906e1ce
Sep 16 13:21:37.048230 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:37.048189    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq" event={"ID":"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5","Type":"ContainerStarted","Data":"c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e"}
Sep 16 13:21:37.048230 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:37.048230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq" event={"ID":"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5","Type":"ContainerStarted","Data":"387fb2adf24b0e17b253a66b70672a6133a4e3a615db2723b551ab87e906e1ce"}
Sep 16 13:21:37.066671 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:37.066607    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq" podStartSLOduration=2.066572322 podStartE2EDuration="2.066572322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:21:37.063296039 +0000 UTC m=+804.740010477" watchObservedRunningTime="2026-09-16 13:21:37.066572322 +0000 UTC m=+804.743286758"
Sep 16 13:21:38.008613 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:38.008561    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff"
Sep 16 13:21:39.013775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:39.013744    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9"
Sep 16 13:21:45.060971 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:45.060937    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:21:45.793214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:45.793176    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:45.793401 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:45.793232    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:45.806379 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:45.806345    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:46.100386 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:46.100314    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:21:53.621930 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.621892    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"]
Sep 16 13:21:53.622364 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.622183    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" podUID="e78cc808-c757-4c63-a497-8124aa121109" containerName="maas-api" containerID="cri-o://83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758" gracePeriod=30
Sep 16 13:21:53.672548 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.672520    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"]
Sep 16 13:21:53.672759 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.672726    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947" podUID="4f119a55-ef58-4aa4-b521-71cc7dc46783" containerName="payload-processing" containerID="cri-o://e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f" gracePeriod=30
Sep 16 13:21:53.686764 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.686734    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"]
Sep 16 13:21:53.686947 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.686927    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx" podUID="7d515a04-bd9a-463c-9bdd-528ef19fad04" containerName="payload-pre-processing" containerID="cri-o://0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb" gracePeriod=30
Sep 16 13:21:53.940199 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.940170    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:53.949335 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.949315    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:53.973704 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:53.973685    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:54.041119 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041092    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnfpx\" (UniqueName: \"kubernetes.io/projected/4f119a55-ef58-4aa4-b521-71cc7dc46783-kube-api-access-jnfpx\") pod \"4f119a55-ef58-4aa4-b521-71cc7dc46783\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") "
Sep 16 13:21:54.041253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041136    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls\") pod \"e78cc808-c757-4c63-a497-8124aa121109\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") "
Sep 16 13:21:54.041253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041171    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-metrics-certs\") pod \"e78cc808-c757-4c63-a497-8124aa121109\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") "
Sep 16 13:21:54.041345 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041328    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm5j5\" (UniqueName: \"kubernetes.io/projected/e78cc808-c757-4c63-a497-8124aa121109-kube-api-access-hm5j5\") pod \"e78cc808-c757-4c63-a497-8124aa121109\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") "
Sep 16 13:21:54.041389 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041374    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f119a55-ef58-4aa4-b521-71cc7dc46783-plugins-config-volume\") pod \"4f119a55-ef58-4aa4-b521-71cc7dc46783\" (UID: \"4f119a55-ef58-4aa4-b521-71cc7dc46783\") "
Sep 16 13:21:54.041439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041405    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78cc808-c757-4c63-a497-8124aa121109-openshift-service-ca\") pod \"e78cc808-c757-4c63-a497-8124aa121109\" (UID: \"e78cc808-c757-4c63-a497-8124aa121109\") "
Sep 16 13:21:54.041888 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041831    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f119a55-ef58-4aa4-b521-71cc7dc46783-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f119a55-ef58-4aa4-b521-71cc7dc46783" (UID: "4f119a55-ef58-4aa4-b521-71cc7dc46783"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:21:54.041888 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.041855    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78cc808-c757-4c63-a497-8124aa121109-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e78cc808-c757-4c63-a497-8124aa121109" (UID: "e78cc808-c757-4c63-a497-8124aa121109"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:21:54.043378 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.043346    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f119a55-ef58-4aa4-b521-71cc7dc46783-kube-api-access-jnfpx" (OuterVolumeSpecName: "kube-api-access-jnfpx") pod "4f119a55-ef58-4aa4-b521-71cc7dc46783" (UID: "4f119a55-ef58-4aa4-b521-71cc7dc46783"). InnerVolumeSpecName "kube-api-access-jnfpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:21:54.043471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.043398    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e78cc808-c757-4c63-a497-8124aa121109" (UID: "e78cc808-c757-4c63-a497-8124aa121109"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:21:54.043471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.043421    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e78cc808-c757-4c63-a497-8124aa121109" (UID: "e78cc808-c757-4c63-a497-8124aa121109"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:21:54.043471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.043455    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e78cc808-c757-4c63-a497-8124aa121109-kube-api-access-hm5j5" (OuterVolumeSpecName: "kube-api-access-hm5j5") pod "e78cc808-c757-4c63-a497-8124aa121109" (UID: "e78cc808-c757-4c63-a497-8124aa121109"). InnerVolumeSpecName "kube-api-access-hm5j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:21:54.134061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.133988    2556 generic.go:356] "Generic (PLEG): container finished" podID="7d515a04-bd9a-463c-9bdd-528ef19fad04" containerID="0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb" exitCode=0
Sep 16 13:21:54.134061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.134038    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"
Sep 16 13:21:54.134244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.134072    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx" event={"ID":"7d515a04-bd9a-463c-9bdd-528ef19fad04","Type":"ContainerDied","Data":"0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb"}
Sep 16 13:21:54.134244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.134129    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx" event={"ID":"7d515a04-bd9a-463c-9bdd-528ef19fad04","Type":"ContainerDied","Data":"5fb3b55610d27cfa0528b65ec6c1dda3d89c20aaa157d78e4030aced48706acb"}
Sep 16 13:21:54.134244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.134157    2556 scope.go:117] "RemoveContainer" containerID="0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb"
Sep 16 13:21:54.135302 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.135279    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f119a55-ef58-4aa4-b521-71cc7dc46783" containerID="e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f" exitCode=0
Sep 16 13:21:54.135383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.135319    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"
Sep 16 13:21:54.135383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.135362    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947" event={"ID":"4f119a55-ef58-4aa4-b521-71cc7dc46783","Type":"ContainerDied","Data":"e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f"}
Sep 16 13:21:54.135475 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.135398    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947" event={"ID":"4f119a55-ef58-4aa4-b521-71cc7dc46783","Type":"ContainerDied","Data":"ac09af6057471de2563eec21c8750607076c5621f066dcfd84fcf9c45a394f79"}
Sep 16 13:21:54.136642 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.136576    2556 generic.go:356] "Generic (PLEG): container finished" podID="e78cc808-c757-4c63-a497-8124aa121109" containerID="83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758" exitCode=0
Sep 16 13:21:54.136642 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.136630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" event={"ID":"e78cc808-c757-4c63-a497-8124aa121109","Type":"ContainerDied","Data":"83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758"}
Sep 16 13:21:54.136772 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.136669    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"
Sep 16 13:21:54.136772 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.136673    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9" event={"ID":"e78cc808-c757-4c63-a497-8124aa121109","Type":"ContainerDied","Data":"33fbba66c14a0572a82a7df1202b2e6c420abdd78ffa3eb19ebf57bcbdc23c96"}
Sep 16 13:21:54.142245 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142222    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d515a04-bd9a-463c-9bdd-528ef19fad04-plugins-config-volume\") pod \"7d515a04-bd9a-463c-9bdd-528ef19fad04\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") "
Sep 16 13:21:54.142355 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142341    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5nl9\" (UniqueName: \"kubernetes.io/projected/7d515a04-bd9a-463c-9bdd-528ef19fad04-kube-api-access-m5nl9\") pod \"7d515a04-bd9a-463c-9bdd-528ef19fad04\" (UID: \"7d515a04-bd9a-463c-9bdd-528ef19fad04\") "
Sep 16 13:21:54.142569 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142549    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnfpx\" (UniqueName: \"kubernetes.io/projected/4f119a55-ef58-4aa4-b521-71cc7dc46783-kube-api-access-jnfpx\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.142569 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142571    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.142734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142584    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78cc808-c757-4c63-a497-8124aa121109-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.142734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142611    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm5j5\" (UniqueName: \"kubernetes.io/projected/e78cc808-c757-4c63-a497-8124aa121109-kube-api-access-hm5j5\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.142734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142627    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f119a55-ef58-4aa4-b521-71cc7dc46783-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.142734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142640    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78cc808-c757-4c63-a497-8124aa121109-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.142734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.142664    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d515a04-bd9a-463c-9bdd-528ef19fad04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d515a04-bd9a-463c-9bdd-528ef19fad04" (UID: "7d515a04-bd9a-463c-9bdd-528ef19fad04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:21:54.144358 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.144333    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d515a04-bd9a-463c-9bdd-528ef19fad04-kube-api-access-m5nl9" (OuterVolumeSpecName: "kube-api-access-m5nl9") pod "7d515a04-bd9a-463c-9bdd-528ef19fad04" (UID: "7d515a04-bd9a-463c-9bdd-528ef19fad04"). InnerVolumeSpecName "kube-api-access-m5nl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:21:54.144531 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.144512    2556 scope.go:117] "RemoveContainer" containerID="0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb"
Sep 16 13:21:54.144891 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:54.144846    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb\": container with ID starting with 0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb not found: ID does not exist" containerID="0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb"
Sep 16 13:21:54.144992 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.144883    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb"} err="failed to get container status \"0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb\": rpc error: code = NotFound desc = could not find container \"0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb\": container with ID starting with 0c7c2dab92f05b2293e31978b25514a57f0a806caf7525f6b0c32c806ac75aeb not found: ID does not exist"
Sep 16 13:21:54.144992 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.144905    2556 scope.go:117] "RemoveContainer" containerID="e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f"
Sep 16 13:21:54.152522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.152507    2556 scope.go:117] "RemoveContainer" containerID="e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f"
Sep 16 13:21:54.152800 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:54.152769    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f\": container with ID starting with e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f not found: ID does not exist" containerID="e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f"
Sep 16 13:21:54.152800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.152794    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f"} err="failed to get container status \"e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f\": rpc error: code = NotFound desc = could not find container \"e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f\": container with ID starting with e137a7e99352475aea4535273fae347167ba5b5f9a314a4eff55669a66ed697f not found: ID does not exist"
Sep 16 13:21:54.152920 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.152809    2556 scope.go:117] "RemoveContainer" containerID="83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758"
Sep 16 13:21:54.160147 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.160128    2556 scope.go:117] "RemoveContainer" containerID="83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758"
Sep 16 13:21:54.160479 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:54.160456    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758\": container with ID starting with 83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758 not found: ID does not exist" containerID="83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758"
Sep 16 13:21:54.160559 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.160484    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758"} err="failed to get container status \"83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758\": rpc error: code = NotFound desc = could not find container \"83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758\": container with ID starting with 83430f9f81d906e4b880bd15430d5c341e30bc45275f12e824ccbedaa83bb758 not found: ID does not exist"
Sep 16 13:21:54.161728 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.161709    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"]
Sep 16 13:21:54.167743 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.167723    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef119741-58f684c-m9947"]
Sep 16 13:21:54.180468 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.180448    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"]
Sep 16 13:21:54.184670 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.184649    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef119741-7c6d89fc79-hg5g9"]
Sep 16 13:21:54.243945 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.243919    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d515a04-bd9a-463c-9bdd-528ef19fad04-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.243945 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.243939    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5nl9\" (UniqueName: \"kubernetes.io/projected/7d515a04-bd9a-463c-9bdd-528ef19fad04-kube-api-access-m5nl9\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:21:54.464880 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.464851    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"]
Sep 16 13:21:54.477838 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.477814    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ef119741-7db79cd8db-jhrrx"]
Sep 16 13:21:54.800251 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.800176    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f119a55-ef58-4aa4-b521-71cc7dc46783" path="/var/lib/kubelet/pods/4f119a55-ef58-4aa4-b521-71cc7dc46783/volumes"
Sep 16 13:21:54.800616 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.800524    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d515a04-bd9a-463c-9bdd-528ef19fad04" path="/var/lib/kubelet/pods/7d515a04-bd9a-463c-9bdd-528ef19fad04/volumes"
Sep 16 13:21:54.800892 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:54.800879    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e78cc808-c757-4c63-a497-8124aa121109" path="/var/lib/kubelet/pods/e78cc808-c757-4c63-a497-8124aa121109/volumes"
Sep 16 13:21:59.299417 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299385    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"]
Sep 16 13:21:59.299789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299718    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d515a04-bd9a-463c-9bdd-528ef19fad04" containerName="payload-pre-processing"
Sep 16 13:21:59.299789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299732    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d515a04-bd9a-463c-9bdd-528ef19fad04" containerName="payload-pre-processing"
Sep 16 13:21:59.299789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299747    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f119a55-ef58-4aa4-b521-71cc7dc46783" containerName="payload-processing"
Sep 16 13:21:59.299789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299752    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f119a55-ef58-4aa4-b521-71cc7dc46783" containerName="payload-processing"
Sep 16 13:21:59.299789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299759    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e78cc808-c757-4c63-a497-8124aa121109" containerName="maas-api"
Sep 16 13:21:59.299789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299764    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e78cc808-c757-4c63-a497-8124aa121109" containerName="maas-api"
Sep 16 13:21:59.299972 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299828    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f119a55-ef58-4aa4-b521-71cc7dc46783" containerName="payload-processing"
Sep 16 13:21:59.299972 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299837    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d515a04-bd9a-463c-9bdd-528ef19fad04" containerName="payload-pre-processing"
Sep 16 13:21:59.299972 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.299846    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e78cc808-c757-4c63-a497-8124aa121109" containerName="maas-api"
Sep 16 13:21:59.302937 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.302915    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.304894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.304875    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-508c5fc8-serving-cert\""
Sep 16 13:21:59.314723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.314703    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"]
Sep 16 13:21:59.480898 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.480871    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc09111-5549-4689-b35d-1c37b165d6fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.481038 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.480917    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sdbz\" (UniqueName: \"kubernetes.io/projected/ecc09111-5549-4689-b35d-1c37b165d6fe-kube-api-access-4sdbz\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.481038 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.480943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.481167 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.481058    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-metrics-certs\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.582367 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.582287    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sdbz\" (UniqueName: \"kubernetes.io/projected/ecc09111-5549-4689-b35d-1c37b165d6fe-kube-api-access-4sdbz\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.582367 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.582329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.582582 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.582373    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-metrics-certs\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.582582 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.582415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc09111-5549-4689-b35d-1c37b165d6fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.582582 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:59.582525    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-serving-cert: secret "maas-api-e2e-migrate-508c5fc8-serving-cert" not found
Sep 16 13:21:59.582732 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:21:59.582636    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls podName:ecc09111-5549-4689-b35d-1c37b165d6fe nodeName:}" failed. No retries permitted until 2026-09-16 13:22:00.082588275 +0000 UTC m=+827.759302688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls") pod "maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" (UID: "ecc09111-5549-4689-b35d-1c37b165d6fe") : secret "maas-api-e2e-migrate-508c5fc8-serving-cert" not found
Sep 16 13:21:59.583037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.583019    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc09111-5549-4689-b35d-1c37b165d6fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.584871 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.584842    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-metrics-certs\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:21:59.593106 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:21:59.593085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sdbz\" (UniqueName: \"kubernetes.io/projected/ecc09111-5549-4689-b35d-1c37b165d6fe-kube-api-access-4sdbz\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:00.070442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.070408    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"]
Sep 16 13:22:00.072775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.072758    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.074891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.074872    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-508c5fc8-dockercfg-qbqk2\""
Sep 16 13:22:00.075371 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.075356    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-508c5fc8\""
Sep 16 13:22:00.084425 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.084404    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"]
Sep 16 13:22:00.087226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.087203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lg2\" (UniqueName: \"kubernetes.io/projected/4fb90116-d950-426d-9646-2f529db68f1b-kube-api-access-67lg2\") pod \"payload-processing-e2e-migrate-508c5fc8-5876c794b-56548\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") " pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.087318 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.087300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:00.087359 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.087335    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb90116-d950-426d-9646-2f529db68f1b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-508c5fc8-5876c794b-56548\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") " pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.089480 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.089458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls\") pod \"maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:00.188346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.188321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lg2\" (UniqueName: \"kubernetes.io/projected/4fb90116-d950-426d-9646-2f529db68f1b-kube-api-access-67lg2\") pod \"payload-processing-e2e-migrate-508c5fc8-5876c794b-56548\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") " pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.188439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.188380    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb90116-d950-426d-9646-2f529db68f1b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-508c5fc8-5876c794b-56548\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") " pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.189007 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.188989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb90116-d950-426d-9646-2f529db68f1b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-508c5fc8-5876c794b-56548\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") " pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.199199 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.199180    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lg2\" (UniqueName: \"kubernetes.io/projected/4fb90116-d950-426d-9646-2f529db68f1b-kube-api-access-67lg2\") pod \"payload-processing-e2e-migrate-508c5fc8-5876c794b-56548\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") " pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.214077 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.214056    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:00.383632 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.383590    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:00.484648 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.484624    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"]
Sep 16 13:22:00.487110 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:22:00.487085    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc09111_5549_4689_b35d_1c37b165d6fe.slice/crio-08e7d6a163c6d4357fc85275c44053e80536d83e13961e903df6f59f6ea1fdab WatchSource:0}: Error finding container 08e7d6a163c6d4357fc85275c44053e80536d83e13961e903df6f59f6ea1fdab: Status 404 returned error can't find the container with id 08e7d6a163c6d4357fc85275c44053e80536d83e13961e903df6f59f6ea1fdab
Sep 16 13:22:00.539765 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.539738    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:22:00.651651 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:00.651568    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"]
Sep 16 13:22:00.654501 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:22:00.654466    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb90116_d950_426d_9646_2f529db68f1b.slice/crio-5ca47149868d3247404b1d9646177528ce59279f92df72cc4a4d1e66b49ba3c2 WatchSource:0}: Error finding container 5ca47149868d3247404b1d9646177528ce59279f92df72cc4a4d1e66b49ba3c2: Status 404 returned error can't find the container with id 5ca47149868d3247404b1d9646177528ce59279f92df72cc4a4d1e66b49ba3c2
Sep 16 13:22:01.173031 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.172998    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548" event={"ID":"4fb90116-d950-426d-9646-2f529db68f1b","Type":"ContainerStarted","Data":"f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a"}
Sep 16 13:22:01.173214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.173060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548" event={"ID":"4fb90116-d950-426d-9646-2f529db68f1b","Type":"ContainerStarted","Data":"5ca47149868d3247404b1d9646177528ce59279f92df72cc4a4d1e66b49ba3c2"}
Sep 16 13:22:01.173214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.173088    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:01.174653 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.174627    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" event={"ID":"ecc09111-5549-4689-b35d-1c37b165d6fe","Type":"ContainerStarted","Data":"e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492"}
Sep 16 13:22:01.174788 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.174667    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" event={"ID":"ecc09111-5549-4689-b35d-1c37b165d6fe","Type":"ContainerStarted","Data":"08e7d6a163c6d4357fc85275c44053e80536d83e13961e903df6f59f6ea1fdab"}
Sep 16 13:22:01.174788 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.174738    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:01.195370 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.195328    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548" podStartSLOduration=1.195314655 podStartE2EDuration="1.195314655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:22:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:22:01.193838394 +0000 UTC m=+828.870552831" watchObservedRunningTime="2026-09-16 13:22:01.195314655 +0000 UTC m=+828.872029122"
Sep 16 13:22:01.217222 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:01.217187    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" podStartSLOduration=1.937319047 podStartE2EDuration="2.217173147s" totalImagesPullingTime="279.8541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:22:01.215663849 +0000 UTC m=+828.892378277" watchObservedRunningTime="2026-09-16 13:22:01.217173147 +0000 UTC m=+828.893887581"
Sep 16 13:22:02.373761 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.373731    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"]
Sep 16 13:22:02.376239 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.376215    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.379135 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.379115    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a21a40\"/\"default-dockercfg-hfk6f\""
Sep 16 13:22:02.379258 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.379132    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a21a40\"/\"openshift-service-ca.crt\""
Sep 16 13:22:02.379258 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.379187    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a21a40\"/\"simulator-w0-a21a40-kserve-self-signed-certs\""
Sep 16 13:22:02.379258 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.379238    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a21a40\"/\"kube-root-ca.crt\""
Sep 16 13:22:02.387633 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.387614    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"]
Sep 16 13:22:02.408400 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.408374    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-home\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.408507 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.408407    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgjbn\" (UniqueName: \"kubernetes.io/projected/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-kube-api-access-mgjbn\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.408507 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.408428    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-model-cache\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.408507 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.408451    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-dshm\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.408641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.408515    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tls-certs\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.408641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.408577    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tmp-dir\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509112 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509087    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tmp-dir\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-home\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509152    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgjbn\" (UniqueName: \"kubernetes.io/projected/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-kube-api-access-mgjbn\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509167    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-model-cache\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509195    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-dshm\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509425 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tls-certs\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509521    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-home\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509523    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tmp-dir\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.509627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.509580    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-model-cache\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.511460 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.511442    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-dshm\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.511656 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.511638    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tls-certs\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.516773 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.516754    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgjbn\" (UniqueName: \"kubernetes.io/projected/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-kube-api-access-mgjbn\") pod \"simulator-w0-a21a40-kserve-6589b5f978-jwzzf\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") " pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.685247 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.685174    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:02.958360 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:02.958299    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"]
Sep 16 13:22:02.960658 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:22:02.960627    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfaaf8faf_de8a_4f8b_acc2_c6c7ded4c9d5.slice/crio-31acc5cdc0e98c6be0f945dd6002f30fb9a88bea65b4a41834380759fa499209 WatchSource:0}: Error finding container 31acc5cdc0e98c6be0f945dd6002f30fb9a88bea65b4a41834380759fa499209: Status 404 returned error can't find the container with id 31acc5cdc0e98c6be0f945dd6002f30fb9a88bea65b4a41834380759fa499209
Sep 16 13:22:03.187141 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:03.187109    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf" event={"ID":"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5","Type":"ContainerStarted","Data":"0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c"}
Sep 16 13:22:03.187291 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:03.187154    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf" event={"ID":"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5","Type":"ContainerStarted","Data":"31acc5cdc0e98c6be0f945dd6002f30fb9a88bea65b4a41834380759fa499209"}
Sep 16 13:22:03.207458 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:03.207417    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf" podStartSLOduration=1.207402339 podStartE2EDuration="1.207402339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:22:03.206441057 +0000 UTC m=+830.883155493" watchObservedRunningTime="2026-09-16 13:22:03.207402339 +0000 UTC m=+830.884116967"
Sep 16 13:22:07.185590 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:07.185556    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:12.182099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:12.182067    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:12.685967 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:12.685931    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:12.686131 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:12.685989    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:12.698549 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:12.698526    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:13.244857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:13.244828    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:22:18.642236 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:18.642199    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:22:28.885064 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:28.885032    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"]
Sep 16 13:22:28.885491 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:28.885229    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" podUID="ecc09111-5549-4689-b35d-1c37b165d6fe" containerName="maas-api" containerID="cri-o://e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492" gracePeriod=30
Sep 16 13:22:28.945755 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:28.945723    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"]
Sep 16 13:22:28.946006 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:28.945970    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548" podUID="4fb90116-d950-426d-9646-2f529db68f1b" containerName="payload-processing" containerID="cri-o://f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a" gracePeriod=30
Sep 16 13:22:29.192181 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.192160    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:29.203068 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.203049    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:29.237700 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.237680    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-metrics-certs\") pod \"ecc09111-5549-4689-b35d-1c37b165d6fe\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") "
Sep 16 13:22:29.237841 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.237711    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc09111-5549-4689-b35d-1c37b165d6fe-openshift-service-ca\") pod \"ecc09111-5549-4689-b35d-1c37b165d6fe\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") "
Sep 16 13:22:29.237841 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.237748    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls\") pod \"ecc09111-5549-4689-b35d-1c37b165d6fe\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") "
Sep 16 13:22:29.237841 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.237822    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sdbz\" (UniqueName: \"kubernetes.io/projected/ecc09111-5549-4689-b35d-1c37b165d6fe-kube-api-access-4sdbz\") pod \"ecc09111-5549-4689-b35d-1c37b165d6fe\" (UID: \"ecc09111-5549-4689-b35d-1c37b165d6fe\") "
Sep 16 13:22:29.238096 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.238073    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc09111-5549-4689-b35d-1c37b165d6fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ecc09111-5549-4689-b35d-1c37b165d6fe" (UID: "ecc09111-5549-4689-b35d-1c37b165d6fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:22:29.239854 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.239826    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ecc09111-5549-4689-b35d-1c37b165d6fe" (UID: "ecc09111-5549-4689-b35d-1c37b165d6fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:22:29.239979 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.239958    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ecc09111-5549-4689-b35d-1c37b165d6fe" (UID: "ecc09111-5549-4689-b35d-1c37b165d6fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:22:29.240039 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.240005    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc09111-5549-4689-b35d-1c37b165d6fe-kube-api-access-4sdbz" (OuterVolumeSpecName: "kube-api-access-4sdbz") pod "ecc09111-5549-4689-b35d-1c37b165d6fe" (UID: "ecc09111-5549-4689-b35d-1c37b165d6fe"). InnerVolumeSpecName "kube-api-access-4sdbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:22:29.305368 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.305336    2556 generic.go:356] "Generic (PLEG): container finished" podID="4fb90116-d950-426d-9646-2f529db68f1b" containerID="f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a" exitCode=0
Sep 16 13:22:29.305522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.305372    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"
Sep 16 13:22:29.305522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.305410    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548" event={"ID":"4fb90116-d950-426d-9646-2f529db68f1b","Type":"ContainerDied","Data":"f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a"}
Sep 16 13:22:29.305522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.305469    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548" event={"ID":"4fb90116-d950-426d-9646-2f529db68f1b","Type":"ContainerDied","Data":"5ca47149868d3247404b1d9646177528ce59279f92df72cc4a4d1e66b49ba3c2"}
Sep 16 13:22:29.305522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.305493    2556 scope.go:117] "RemoveContainer" containerID="f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a"
Sep 16 13:22:29.306797 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.306776    2556 generic.go:356] "Generic (PLEG): container finished" podID="ecc09111-5549-4689-b35d-1c37b165d6fe" containerID="e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492" exitCode=0
Sep 16 13:22:29.306902 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.306820    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"
Sep 16 13:22:29.306902 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.306844    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" event={"ID":"ecc09111-5549-4689-b35d-1c37b165d6fe","Type":"ContainerDied","Data":"e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492"}
Sep 16 13:22:29.306902 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.306873    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5" event={"ID":"ecc09111-5549-4689-b35d-1c37b165d6fe","Type":"ContainerDied","Data":"08e7d6a163c6d4357fc85275c44053e80536d83e13961e903df6f59f6ea1fdab"}
Sep 16 13:22:29.314519 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.314495    2556 scope.go:117] "RemoveContainer" containerID="f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a"
Sep 16 13:22:29.314823 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:22:29.314801    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a\": container with ID starting with f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a not found: ID does not exist" containerID="f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a"
Sep 16 13:22:29.314894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.314830    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a"} err="failed to get container status \"f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a\": rpc error: code = NotFound desc = could not find container \"f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a\": container with ID starting with f72cc7b0103fcf9f790f8cf32d902ea66d10d6af037f328c5ebb9b756efa234a not found: ID does not exist"
Sep 16 13:22:29.314894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.314846    2556 scope.go:117] "RemoveContainer" containerID="e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492"
Sep 16 13:22:29.328168 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.328142    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"]
Sep 16 13:22:29.329469 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.329449    2556 scope.go:117] "RemoveContainer" containerID="e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492"
Sep 16 13:22:29.329775 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:22:29.329756    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492\": container with ID starting with e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492 not found: ID does not exist" containerID="e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492"
Sep 16 13:22:29.329833 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.329782    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492"} err="failed to get container status \"e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492\": rpc error: code = NotFound desc = could not find container \"e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492\": container with ID starting with e413c226c4c24396bc54c1f8f408d31c0bd5f1fc7818d388a332bf66df182492 not found: ID does not exist"
Sep 16 13:22:29.332152 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.332131    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-508c5fc8-59bc74dc94-q2ds5"]
Sep 16 13:22:29.338739 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.338714    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67lg2\" (UniqueName: \"kubernetes.io/projected/4fb90116-d950-426d-9646-2f529db68f1b-kube-api-access-67lg2\") pod \"4fb90116-d950-426d-9646-2f529db68f1b\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") "
Sep 16 13:22:29.338870 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.338800    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb90116-d950-426d-9646-2f529db68f1b-plugins-config-volume\") pod \"4fb90116-d950-426d-9646-2f529db68f1b\" (UID: \"4fb90116-d950-426d-9646-2f529db68f1b\") "
Sep 16 13:22:29.339054 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.339038    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:22:29.339111 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.339061    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sdbz\" (UniqueName: \"kubernetes.io/projected/ecc09111-5549-4689-b35d-1c37b165d6fe-kube-api-access-4sdbz\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:22:29.339111 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.339075    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc09111-5549-4689-b35d-1c37b165d6fe-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:22:29.339111 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.339089    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc09111-5549-4689-b35d-1c37b165d6fe-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:22:29.339111 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.339094    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb90116-d950-426d-9646-2f529db68f1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4fb90116-d950-426d-9646-2f529db68f1b" (UID: "4fb90116-d950-426d-9646-2f529db68f1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:22:29.340482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.340461    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb90116-d950-426d-9646-2f529db68f1b-kube-api-access-67lg2" (OuterVolumeSpecName: "kube-api-access-67lg2") pod "4fb90116-d950-426d-9646-2f529db68f1b" (UID: "4fb90116-d950-426d-9646-2f529db68f1b"). InnerVolumeSpecName "kube-api-access-67lg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:22:29.440097 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.440050    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67lg2\" (UniqueName: \"kubernetes.io/projected/4fb90116-d950-426d-9646-2f529db68f1b-kube-api-access-67lg2\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:22:29.440097 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.440071    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb90116-d950-426d-9646-2f529db68f1b-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:22:29.624946 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.624916    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"]
Sep 16 13:22:29.628997 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:29.628973    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-508c5fc8-5876c794b-56548"]
Sep 16 13:22:30.801167 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:30.801120    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb90116-d950-426d-9646-2f529db68f1b" path="/var/lib/kubelet/pods/4fb90116-d950-426d-9646-2f529db68f1b/volumes"
Sep 16 13:22:30.801743 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:30.801720    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecc09111-5549-4689-b35d-1c37b165d6fe" path="/var/lib/kubelet/pods/ecc09111-5549-4689-b35d-1c37b165d6fe/volumes"
Sep 16 13:22:36.848152 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:36.848118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:22:37.507699 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.507663    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"]
Sep 16 13:22:37.507992 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.507978    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb90116-d950-426d-9646-2f529db68f1b" containerName="payload-processing"
Sep 16 13:22:37.508044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.507994    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb90116-d950-426d-9646-2f529db68f1b" containerName="payload-processing"
Sep 16 13:22:37.508044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.508005    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecc09111-5549-4689-b35d-1c37b165d6fe" containerName="maas-api"
Sep 16 13:22:37.508044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.508011    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecc09111-5549-4689-b35d-1c37b165d6fe" containerName="maas-api"
Sep 16 13:22:37.508157 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.508097    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb90116-d950-426d-9646-2f529db68f1b" containerName="payload-processing"
Sep 16 13:22:37.508157 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.508107    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecc09111-5549-4689-b35d-1c37b165d6fe" containerName="maas-api"
Sep 16 13:22:37.511220 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.511192    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.513621 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.513583    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fa486478-serving-cert\""
Sep 16 13:22:37.524806 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.524781    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"]
Sep 16 13:22:37.605833 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.605804    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt9p8\" (UniqueName: \"kubernetes.io/projected/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-kube-api-access-qt9p8\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.605938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.605849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-openshift-service-ca\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.605938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.605866    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.606013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.605994    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-metrics-certs\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.707319 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.707290    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt9p8\" (UniqueName: \"kubernetes.io/projected/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-kube-api-access-qt9p8\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.707427 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.707341    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-openshift-service-ca\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.707427 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.707368    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.707538 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:22:37.707444    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-serving-cert: secret "maas-api-e2e-ait-fa486478-serving-cert" not found
Sep 16 13:22:37.707538 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.707461    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-metrics-certs\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.707538 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:22:37.707504    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls podName:84fa7cb4-0bd7-4302-8b34-3ff3eed26243 nodeName:}" failed. No retries permitted until 2026-09-16 13:22:38.207489103 +0000 UTC m=+865.884203516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls") pod "maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" (UID: "84fa7cb4-0bd7-4302-8b34-3ff3eed26243") : secret "maas-api-e2e-ait-fa486478-serving-cert" not found
Sep 16 13:22:37.707896 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.707876    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-openshift-service-ca\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.709751 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.709733    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-metrics-certs\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:37.715937 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:37.715914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt9p8\" (UniqueName: \"kubernetes.io/projected/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-kube-api-access-qt9p8\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:38.167441 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.167408    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"]
Sep 16 13:22:38.169878 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.169858    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.172534 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.172513    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fa486478\""
Sep 16 13:22:38.172617 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.172555    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fa486478-dockercfg-gl22s\""
Sep 16 13:22:38.188614 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.188579    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"]
Sep 16 13:22:38.212893 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.212872    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:38.215242 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.215219    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls\") pod \"maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:38.313914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.313884    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw84g\" (UniqueName: \"kubernetes.io/projected/2dd75a84-7001-4393-8abb-86f6571611c3-kube-api-access-lw84g\") pod \"payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") " pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.314011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.313957    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd75a84-7001-4393-8abb-86f6571611c3-plugins-config-volume\") pod \"payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") " pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.414702 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.414678    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw84g\" (UniqueName: \"kubernetes.io/projected/2dd75a84-7001-4393-8abb-86f6571611c3-kube-api-access-lw84g\") pod \"payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") " pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.414800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.414776    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd75a84-7001-4393-8abb-86f6571611c3-plugins-config-volume\") pod \"payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") " pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.415379 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.415363    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd75a84-7001-4393-8abb-86f6571611c3-plugins-config-volume\") pod \"payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") " pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.421719 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.421676    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:38.422024 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.422009    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw84g\" (UniqueName: \"kubernetes.io/projected/2dd75a84-7001-4393-8abb-86f6571611c3-kube-api-access-lw84g\") pod \"payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") " pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.480209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.480114    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:38.692745 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.692672    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"]
Sep 16 13:22:38.695568 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:22:38.695538    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84fa7cb4_0bd7_4302_8b34_3ff3eed26243.slice/crio-8e71537bcb7cb0868708cdca5afc97b38d399a96caa67aeca17a9fc32ef70e21 WatchSource:0}: Error finding container 8e71537bcb7cb0868708cdca5afc97b38d399a96caa67aeca17a9fc32ef70e21: Status 404 returned error can't find the container with id 8e71537bcb7cb0868708cdca5afc97b38d399a96caa67aeca17a9fc32ef70e21
Sep 16 13:22:38.752448 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:38.752426    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"]
Sep 16 13:22:38.754444 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:22:38.754420    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd75a84_7001_4393_8abb_86f6571611c3.slice/crio-423186f5c0eb6b4fe9c0b51e19f5914fa360c5ef098505cdc7c99f01991ab386 WatchSource:0}: Error finding container 423186f5c0eb6b4fe9c0b51e19f5914fa360c5ef098505cdc7c99f01991ab386: Status 404 returned error can't find the container with id 423186f5c0eb6b4fe9c0b51e19f5914fa360c5ef098505cdc7c99f01991ab386
Sep 16 13:22:39.356099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.356012    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9" event={"ID":"2dd75a84-7001-4393-8abb-86f6571611c3","Type":"ContainerStarted","Data":"e65ae6cab37eb0ac984986a332072c5ca1eb9ab3b96d540057f0018c9d06ea72"}
Sep 16 13:22:39.356099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.356070    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9" event={"ID":"2dd75a84-7001-4393-8abb-86f6571611c3","Type":"ContainerStarted","Data":"423186f5c0eb6b4fe9c0b51e19f5914fa360c5ef098505cdc7c99f01991ab386"}
Sep 16 13:22:39.356561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.356103    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:39.357897 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.357869    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" event={"ID":"84fa7cb4-0bd7-4302-8b34-3ff3eed26243","Type":"ContainerStarted","Data":"c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd"}
Sep 16 13:22:39.358016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.357911    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" event={"ID":"84fa7cb4-0bd7-4302-8b34-3ff3eed26243","Type":"ContainerStarted","Data":"8e71537bcb7cb0868708cdca5afc97b38d399a96caa67aeca17a9fc32ef70e21"}
Sep 16 13:22:39.358016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.358002    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:39.373453 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.373404    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9" podStartSLOduration=1.3733852610000001 podStartE2EDuration="1.373385261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:22:39.371342543 +0000 UTC m=+867.048056979" watchObservedRunningTime="2026-09-16 13:22:39.373385261 +0000 UTC m=+867.050099698"
Sep 16 13:22:39.396617 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:39.391858    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" podStartSLOduration=2.013933819 podStartE2EDuration="2.391840393s" totalImagesPullingTime="377.906574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:22:39.389447061 +0000 UTC m=+867.066161496" watchObservedRunningTime="2026-09-16 13:22:39.391840393 +0000 UTC m=+867.068554830"
Sep 16 13:22:45.368860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:45.368830    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:22:50.366093 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:50.366063    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:22:52.155715 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:22:52.155686    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:23:09.382497 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.382420    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"]
Sep 16 13:23:09.385652 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.385631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.388420 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.388399    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-15da7b\"/\"kube-root-ca.crt\""
Sep 16 13:23:09.388420 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.388410    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-15da7b\"/\"default-dockercfg-f8zsc\""
Sep 16 13:23:09.388589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.388424    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-15da7b\"/\"openshift-service-ca.crt\""
Sep 16 13:23:09.388589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.388408    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-15da7b\"/\"autores-model-15da7b-kserve-self-signed-certs\""
Sep 16 13:23:09.396240 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.396166    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"]
Sep 16 13:23:09.466834 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.466802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-dshm\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.466936 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.466850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-model-cache\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.466936 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.466888    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tt5w\" (UniqueName: \"kubernetes.io/projected/b659bb4c-5550-49f0-9370-8b71400c9167-kube-api-access-9tt5w\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.467036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.467021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-home\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.467080 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.467050    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b659bb4c-5550-49f0-9370-8b71400c9167-tls-certs\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.467128 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.467086    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-tmp-dir\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568272 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568238    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-home\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568272 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568274    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b659bb4c-5550-49f0-9370-8b71400c9167-tls-certs\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568304    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-tmp-dir\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-dshm\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-model-cache\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568490 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tt5w\" (UniqueName: \"kubernetes.io/projected/b659bb4c-5550-49f0-9370-8b71400c9167-kube-api-access-9tt5w\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568739 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568718    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-home\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568818 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568785    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-tmp-dir\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.568871 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.568813    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-model-cache\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.570690 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.570665    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-dshm\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.570878 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.570860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b659bb4c-5550-49f0-9370-8b71400c9167-tls-certs\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.577257 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.577233    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tt5w\" (UniqueName: \"kubernetes.io/projected/b659bb4c-5550-49f0-9370-8b71400c9167-kube-api-access-9tt5w\") pod \"autores-model-15da7b-kserve-684bcdc8dc-8zpww\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") " pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.720649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.720549    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:09.990086 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:09.990061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"]
Sep 16 13:23:09.992728 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:23:09.992701    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb659bb4c_5550_49f0_9370_8b71400c9167.slice/crio-7bbe26251296eb4ca8000708349f1688a536100a64bdaf2f829a5ce1663a1464 WatchSource:0}: Error finding container 7bbe26251296eb4ca8000708349f1688a536100a64bdaf2f829a5ce1663a1464: Status 404 returned error can't find the container with id 7bbe26251296eb4ca8000708349f1688a536100a64bdaf2f829a5ce1663a1464
Sep 16 13:23:10.508264 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:10.508228    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww" event={"ID":"b659bb4c-5550-49f0-9370-8b71400c9167","Type":"ContainerStarted","Data":"3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f"}
Sep 16 13:23:10.508264 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:10.508271    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww" event={"ID":"b659bb4c-5550-49f0-9370-8b71400c9167","Type":"ContainerStarted","Data":"7bbe26251296eb4ca8000708349f1688a536100a64bdaf2f829a5ce1663a1464"}
Sep 16 13:23:10.526518 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:10.526472    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww" podStartSLOduration=1.526456094 podStartE2EDuration="1.526456094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:23:10.525246235 +0000 UTC m=+898.201960665" watchObservedRunningTime="2026-09-16 13:23:10.526456094 +0000 UTC m=+898.203170566"
Sep 16 13:23:12.805728 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:12.805702    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:23:12.810665 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:12.810642    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:23:15.189339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.189307    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"]
Sep 16 13:23:15.189812 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.189573    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" podUID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" containerName="maas-api" containerID="cri-o://c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd" gracePeriod=30
Sep 16 13:23:15.259121 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.258535    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:23:15.264858 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.264831    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"]
Sep 16 13:23:15.265062 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.265038    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9" podUID="2dd75a84-7001-4393-8abb-86f6571611c3" containerName="payload-processing" containerID="cri-o://e65ae6cab37eb0ac984986a332072c5ca1eb9ab3b96d540057f0018c9d06ea72" gracePeriod=30
Sep 16 13:23:15.364362 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.364322    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" podUID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.43:8443/health\": dial tcp 10.132.0.43:8443: connect: connection refused"
Sep 16 13:23:15.522019 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.521989    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:23:15.534242 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.534214    2556 generic.go:356] "Generic (PLEG): container finished" podID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" containerID="c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd" exitCode=0
Sep 16 13:23:15.534387 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.534258    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"
Sep 16 13:23:15.534387 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.534309    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" event={"ID":"84fa7cb4-0bd7-4302-8b34-3ff3eed26243","Type":"ContainerDied","Data":"c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd"}
Sep 16 13:23:15.534387 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.534363    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt" event={"ID":"84fa7cb4-0bd7-4302-8b34-3ff3eed26243","Type":"ContainerDied","Data":"8e71537bcb7cb0868708cdca5afc97b38d399a96caa67aeca17a9fc32ef70e21"}
Sep 16 13:23:15.534746 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.534397    2556 scope.go:117] "RemoveContainer" containerID="c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd"
Sep 16 13:23:15.536068 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.535967    2556 generic.go:356] "Generic (PLEG): container finished" podID="2dd75a84-7001-4393-8abb-86f6571611c3" containerID="e65ae6cab37eb0ac984986a332072c5ca1eb9ab3b96d540057f0018c9d06ea72" exitCode=0
Sep 16 13:23:15.536068 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.536049    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9" event={"ID":"2dd75a84-7001-4393-8abb-86f6571611c3","Type":"ContainerDied","Data":"e65ae6cab37eb0ac984986a332072c5ca1eb9ab3b96d540057f0018c9d06ea72"}
Sep 16 13:23:15.546086 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.546066    2556 scope.go:117] "RemoveContainer" containerID="c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd"
Sep 16 13:23:15.546455 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:23:15.546429    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd\": container with ID starting with c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd not found: ID does not exist" containerID="c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd"
Sep 16 13:23:15.546568 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.546465    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd"} err="failed to get container status \"c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd\": rpc error: code = NotFound desc = could not find container \"c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd\": container with ID starting with c706e7d12561d59a42643704b8b66667f3dcb596fbdb9b931cd3e8e81087d0fd not found: ID does not exist"
Sep 16 13:23:15.547282 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.547262    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:23:15.617755 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.617717    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-openshift-service-ca\") pod \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") "
Sep 16 13:23:15.617935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.617770    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd75a84-7001-4393-8abb-86f6571611c3-plugins-config-volume\") pod \"2dd75a84-7001-4393-8abb-86f6571611c3\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") "
Sep 16 13:23:15.617935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.617810    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt9p8\" (UniqueName: \"kubernetes.io/projected/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-kube-api-access-qt9p8\") pod \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") "
Sep 16 13:23:15.617935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.617891    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-metrics-certs\") pod \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") "
Sep 16 13:23:15.617935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.617916    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw84g\" (UniqueName: \"kubernetes.io/projected/2dd75a84-7001-4393-8abb-86f6571611c3-kube-api-access-lw84g\") pod \"2dd75a84-7001-4393-8abb-86f6571611c3\" (UID: \"2dd75a84-7001-4393-8abb-86f6571611c3\") "
Sep 16 13:23:15.618152 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.618062    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls\") pod \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\" (UID: \"84fa7cb4-0bd7-4302-8b34-3ff3eed26243\") "
Sep 16 13:23:15.618152 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.618117    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84fa7cb4-0bd7-4302-8b34-3ff3eed26243" (UID: "84fa7cb4-0bd7-4302-8b34-3ff3eed26243"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:23:15.618288 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.618149    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dd75a84-7001-4393-8abb-86f6571611c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2dd75a84-7001-4393-8abb-86f6571611c3" (UID: "2dd75a84-7001-4393-8abb-86f6571611c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:23:15.618763 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.618738    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:23:15.618876 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.618767    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd75a84-7001-4393-8abb-86f6571611c3-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:23:15.620040 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.620016    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-kube-api-access-qt9p8" (OuterVolumeSpecName: "kube-api-access-qt9p8") pod "84fa7cb4-0bd7-4302-8b34-3ff3eed26243" (UID: "84fa7cb4-0bd7-4302-8b34-3ff3eed26243"). InnerVolumeSpecName "kube-api-access-qt9p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:23:15.620293 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.620265    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84fa7cb4-0bd7-4302-8b34-3ff3eed26243" (UID: "84fa7cb4-0bd7-4302-8b34-3ff3eed26243"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:23:15.620382 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.620361    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "84fa7cb4-0bd7-4302-8b34-3ff3eed26243" (UID: "84fa7cb4-0bd7-4302-8b34-3ff3eed26243"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:23:15.620467 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.620442    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd75a84-7001-4393-8abb-86f6571611c3-kube-api-access-lw84g" (OuterVolumeSpecName: "kube-api-access-lw84g") pod "2dd75a84-7001-4393-8abb-86f6571611c3" (UID: "2dd75a84-7001-4393-8abb-86f6571611c3"). InnerVolumeSpecName "kube-api-access-lw84g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:23:15.719674 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.719566    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:23:15.719674 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.719610    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt9p8\" (UniqueName: \"kubernetes.io/projected/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-kube-api-access-qt9p8\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:23:15.719674 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.719626    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84fa7cb4-0bd7-4302-8b34-3ff3eed26243-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:23:15.719674 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.719639    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw84g\" (UniqueName: \"kubernetes.io/projected/2dd75a84-7001-4393-8abb-86f6571611c3-kube-api-access-lw84g\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:23:15.868588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.868557    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"]
Sep 16 13:23:15.880015 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:15.879976    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fa486478-7d8c679b49-mcxgt"]
Sep 16 13:23:16.542546 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.542469    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"
Sep 16 13:23:16.542546 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.542495    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9" event={"ID":"2dd75a84-7001-4393-8abb-86f6571611c3","Type":"ContainerDied","Data":"423186f5c0eb6b4fe9c0b51e19f5914fa360c5ef098505cdc7c99f01991ab386"}
Sep 16 13:23:16.543020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.542558    2556 scope.go:117] "RemoveContainer" containerID="e65ae6cab37eb0ac984986a332072c5ca1eb9ab3b96d540057f0018c9d06ea72"
Sep 16 13:23:16.562962 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.562934    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"]
Sep 16 13:23:16.565308 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.565285    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fa486478-54f6467c6c-4wxj9"]
Sep 16 13:23:16.800774 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.800699    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd75a84-7001-4393-8abb-86f6571611c3" path="/var/lib/kubelet/pods/2dd75a84-7001-4393-8abb-86f6571611c3/volumes"
Sep 16 13:23:16.801109 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:16.801096    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" path="/var/lib/kubelet/pods/84fa7cb4-0bd7-4302-8b34-3ff3eed26243/volumes"
Sep 16 13:23:19.721203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:19.721170    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:19.721673 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:19.721215    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:19.733243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:19.733222    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:20.572843 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:20.572815    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:23:25.311040 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311005    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"]
Sep 16 13:23:25.311443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311354    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" containerName="maas-api"
Sep 16 13:23:25.311443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311367    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" containerName="maas-api"
Sep 16 13:23:25.311443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311383    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd75a84-7001-4393-8abb-86f6571611c3" containerName="payload-processing"
Sep 16 13:23:25.311443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311390    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd75a84-7001-4393-8abb-86f6571611c3" containerName="payload-processing"
Sep 16 13:23:25.311581 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311458    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="84fa7cb4-0bd7-4302-8b34-3ff3eed26243" containerName="maas-api"
Sep 16 13:23:25.311581 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.311467    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd75a84-7001-4393-8abb-86f6571611c3" containerName="payload-processing"
Sep 16 13:23:25.316088 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.316060    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.318222 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.318195    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9615f1d6-serving-cert\""
Sep 16 13:23:25.327714 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.327688    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"]
Sep 16 13:23:25.403564 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.403536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6069cd-5736-4b83-89fd-f4b0cea430ca-openshift-service-ca\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.403564 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.403566    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j88ls\" (UniqueName: \"kubernetes.io/projected/0b6069cd-5736-4b83-89fd-f4b0cea430ca-kube-api-access-j88ls\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.403758 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.403614    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-metrics-certs\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.403758 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.403710    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-maas-api-tls\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.505022 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.504990    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-maas-api-tls\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.505158 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.505092    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6069cd-5736-4b83-89fd-f4b0cea430ca-openshift-service-ca\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.505158 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.505122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j88ls\" (UniqueName: \"kubernetes.io/projected/0b6069cd-5736-4b83-89fd-f4b0cea430ca-kube-api-access-j88ls\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.505276 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.505166    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-metrics-certs\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.505757 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.505735    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6069cd-5736-4b83-89fd-f4b0cea430ca-openshift-service-ca\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.507558 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.507536    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-maas-api-tls\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.507664 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.507568    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-metrics-certs\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.512658 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.512637    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j88ls\" (UniqueName: \"kubernetes.io/projected/0b6069cd-5736-4b83-89fd-f4b0cea430ca-kube-api-access-j88ls\") pod \"maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.628109 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.628085    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:25.902195 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:25.902171    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"]
Sep 16 13:23:25.903810 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:23:25.903783    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6069cd_5736_4b83_89fd_f4b0cea430ca.slice/crio-6460412f692fd35a89ce863be748454005351d91f150bd0d5e079396641eb3f2 WatchSource:0}: Error finding container 6460412f692fd35a89ce863be748454005351d91f150bd0d5e079396641eb3f2: Status 404 returned error can't find the container with id 6460412f692fd35a89ce863be748454005351d91f150bd0d5e079396641eb3f2
Sep 16 13:23:26.058805 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.058773    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"]
Sep 16 13:23:26.063127 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.063112    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.065668 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.065645    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9615f1d6\""
Sep 16 13:23:26.065744 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.065690    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9615f1d6-dockercfg-2lgh6\""
Sep 16 13:23:26.072911 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.072888    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"]
Sep 16 13:23:26.211606 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.211518    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4a3a584-03a3-4d4e-a034-b70f6b640109-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.211723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.211621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgs9h\" (UniqueName: \"kubernetes.io/projected/c4a3a584-03a3-4d4e-a034-b70f6b640109-kube-api-access-cgs9h\") pod \"payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.312692 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.312668    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4a3a584-03a3-4d4e-a034-b70f6b640109-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.312985 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.312704    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgs9h\" (UniqueName: \"kubernetes.io/projected/c4a3a584-03a3-4d4e-a034-b70f6b640109-kube-api-access-cgs9h\") pod \"payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.313391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.313371    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4a3a584-03a3-4d4e-a034-b70f6b640109-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.320292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.320274    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgs9h\" (UniqueName: \"kubernetes.io/projected/c4a3a584-03a3-4d4e-a034-b70f6b640109-kube-api-access-cgs9h\") pod \"payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.373718 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.373696    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:26.595500 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.595419    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b" event={"ID":"0b6069cd-5736-4b83-89fd-f4b0cea430ca","Type":"ContainerStarted","Data":"008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f"}
Sep 16 13:23:26.595500 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.595478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b" event={"ID":"0b6069cd-5736-4b83-89fd-f4b0cea430ca","Type":"ContainerStarted","Data":"6460412f692fd35a89ce863be748454005351d91f150bd0d5e079396641eb3f2"}
Sep 16 13:23:26.595708 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.595518    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:26.613175 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.613128    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b" podStartSLOduration=1.198239492 podStartE2EDuration="1.613113821s" totalImagesPullingTime="414.874329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:23:26.61224415 +0000 UTC m=+914.288958582" watchObservedRunningTime="2026-09-16 13:23:26.613113821 +0000 UTC m=+914.289828255"
Sep 16 13:23:26.643752 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:26.643731    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"]
Sep 16 13:23:26.645615 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:23:26.645573    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a3a584_03a3_4d4e_a034_b70f6b640109.slice/crio-116483a7f99cab6c74cc89f2bfc6e5d8d0b5521dbd29084bdd307525b0f6c9eb WatchSource:0}: Error finding container 116483a7f99cab6c74cc89f2bfc6e5d8d0b5521dbd29084bdd307525b0f6c9eb: Status 404 returned error can't find the container with id 116483a7f99cab6c74cc89f2bfc6e5d8d0b5521dbd29084bdd307525b0f6c9eb
Sep 16 13:23:27.601732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:27.601691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t" event={"ID":"c4a3a584-03a3-4d4e-a034-b70f6b640109","Type":"ContainerStarted","Data":"34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395"}
Sep 16 13:23:27.602126 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:27.601755    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t" event={"ID":"c4a3a584-03a3-4d4e-a034-b70f6b640109","Type":"ContainerStarted","Data":"116483a7f99cab6c74cc89f2bfc6e5d8d0b5521dbd29084bdd307525b0f6c9eb"}
Sep 16 13:23:27.602126 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:27.601984    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:27.619450 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:27.619410    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t" podStartSLOduration=1.6193957060000002 podStartE2EDuration="1.619395706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:23:27.617411303 +0000 UTC m=+915.294125738" watchObservedRunningTime="2026-09-16 13:23:27.619395706 +0000 UTC m=+915.296110138"
Sep 16 13:23:31.584912 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.584875    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"]
Sep 16 13:23:31.587993 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.587978    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.590340 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.590319    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-48adf5\"/\"openshift-service-ca.crt\""
Sep 16 13:23:31.591141 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.591121    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-48adf5\"/\"test-model-48adf5-kserve-self-signed-certs\""
Sep 16 13:23:31.591204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.591139    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-48adf5\"/\"kube-root-ca.crt\""
Sep 16 13:23:31.591204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.591131    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-48adf5\"/\"default-dockercfg-mvlzq\""
Sep 16 13:23:31.596850 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.596818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"]
Sep 16 13:23:31.658474 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.658445    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-tmp-dir\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.658474 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.658475    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-dshm\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.658685 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.658492    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/181a10db-31ec-46ae-8359-b29d99fbdace-tls-certs\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.658685 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.658544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-model-cache\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.658685 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.658615    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-home\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.658685 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.658643    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnk6\" (UniqueName: \"kubernetes.io/projected/181a10db-31ec-46ae-8359-b29d99fbdace-kube-api-access-wjnk6\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759539 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759516    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-model-cache\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759696 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759546    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-home\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759696 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759561    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnk6\" (UniqueName: \"kubernetes.io/projected/181a10db-31ec-46ae-8359-b29d99fbdace-kube-api-access-wjnk6\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759696 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-tmp-dir\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759696 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759678    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-dshm\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759921 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759776    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/181a10db-31ec-46ae-8359-b29d99fbdace-tls-certs\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.759980 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759917    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-model-cache\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.760040 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.759997    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-home\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.760084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.760065    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-tmp-dir\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.761747 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.761719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-dshm\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.762160 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.762142    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/181a10db-31ec-46ae-8359-b29d99fbdace-tls-certs\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.766949 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.766929    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnk6\" (UniqueName: \"kubernetes.io/projected/181a10db-31ec-46ae-8359-b29d99fbdace-kube-api-access-wjnk6\") pod \"test-model-48adf5-kserve-6c78f9bdd5-nb2vz\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") " pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:31.898365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:31.898344    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:32.167984 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:32.167962    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"]
Sep 16 13:23:32.170687 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:23:32.170654    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod181a10db_31ec_46ae_8359_b29d99fbdace.slice/crio-86ee347ee2feccf40ad52d3ee1e13f760c85c937ee521bb86483ac32bebafc62 WatchSource:0}: Error finding container 86ee347ee2feccf40ad52d3ee1e13f760c85c937ee521bb86483ac32bebafc62: Status 404 returned error can't find the container with id 86ee347ee2feccf40ad52d3ee1e13f760c85c937ee521bb86483ac32bebafc62
Sep 16 13:23:32.607151 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:32.607064    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:23:32.629310 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:32.629264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz" event={"ID":"181a10db-31ec-46ae-8359-b29d99fbdace","Type":"ContainerStarted","Data":"ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3"}
Sep 16 13:23:32.629310 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:32.629308    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz" event={"ID":"181a10db-31ec-46ae-8359-b29d99fbdace","Type":"ContainerStarted","Data":"86ee347ee2feccf40ad52d3ee1e13f760c85c937ee521bb86483ac32bebafc62"}
Sep 16 13:23:32.649863 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:32.649805    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz" podStartSLOduration=1.649788968 podStartE2EDuration="1.649788968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:23:32.648342863 +0000 UTC m=+920.325057322" watchObservedRunningTime="2026-09-16 13:23:32.649788968 +0000 UTC m=+920.326503402"
Sep 16 13:23:33.854168 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:33.854132    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:23:38.609158 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:38.609127    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:23:41.899239 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:41.899206    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:41.899239 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:41.899245    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:41.914749 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:41.914720    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:42.697855 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:42.697808    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:23:49.044909 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:23:49.044871    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:24:02.291332 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:02.291296    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"]
Sep 16 13:24:02.790406 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:02.790370    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"]
Sep 16 13:24:02.790582 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:02.790561    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b" podUID="0b6069cd-5736-4b83-89fd-f4b0cea430ca" containerName="maas-api" containerID="cri-o://008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f" gracePeriod=30
Sep 16 13:24:02.913639 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:02.913609    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"]
Sep 16 13:24:02.913882 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:02.913855    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t" podUID="c4a3a584-03a3-4d4e-a034-b70f6b640109" containerName="payload-pre-processing" containerID="cri-o://34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395" gracePeriod=30
Sep 16 13:24:03.050261 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.050201    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:24:03.109170 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.109129    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-maas-api-tls\") pod \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") "
Sep 16 13:24:03.109313 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.109273    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j88ls\" (UniqueName: \"kubernetes.io/projected/0b6069cd-5736-4b83-89fd-f4b0cea430ca-kube-api-access-j88ls\") pod \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") "
Sep 16 13:24:03.109363 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.109310    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6069cd-5736-4b83-89fd-f4b0cea430ca-openshift-service-ca\") pod \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") "
Sep 16 13:24:03.109404 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.109373    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-metrics-certs\") pod \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\" (UID: \"0b6069cd-5736-4b83-89fd-f4b0cea430ca\") "
Sep 16 13:24:03.109779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.109717    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b6069cd-5736-4b83-89fd-f4b0cea430ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0b6069cd-5736-4b83-89fd-f4b0cea430ca" (UID: "0b6069cd-5736-4b83-89fd-f4b0cea430ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:24:03.112583 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.112551    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0b6069cd-5736-4b83-89fd-f4b0cea430ca" (UID: "0b6069cd-5736-4b83-89fd-f4b0cea430ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:24:03.112792 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.112669    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0b6069cd-5736-4b83-89fd-f4b0cea430ca" (UID: "0b6069cd-5736-4b83-89fd-f4b0cea430ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:24:03.113013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.112982    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6069cd-5736-4b83-89fd-f4b0cea430ca-kube-api-access-j88ls" (OuterVolumeSpecName: "kube-api-access-j88ls") pod "0b6069cd-5736-4b83-89fd-f4b0cea430ca" (UID: "0b6069cd-5736-4b83-89fd-f4b0cea430ca"). InnerVolumeSpecName "kube-api-access-j88ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:24:03.164006 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.163983    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:24:03.210744 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.210711    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4a3a584-03a3-4d4e-a034-b70f6b640109-plugins-config-volume\") pod \"c4a3a584-03a3-4d4e-a034-b70f6b640109\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") "
Sep 16 13:24:03.210876 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.210785    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgs9h\" (UniqueName: \"kubernetes.io/projected/c4a3a584-03a3-4d4e-a034-b70f6b640109-kube-api-access-cgs9h\") pod \"c4a3a584-03a3-4d4e-a034-b70f6b640109\" (UID: \"c4a3a584-03a3-4d4e-a034-b70f6b640109\") "
Sep 16 13:24:03.210992 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.210979    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6069cd-5736-4b83-89fd-f4b0cea430ca-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:03.211058 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.210992    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:03.211058 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.211002    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6069cd-5736-4b83-89fd-f4b0cea430ca-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:03.211058 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.211012    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j88ls\" (UniqueName: \"kubernetes.io/projected/0b6069cd-5736-4b83-89fd-f4b0cea430ca-kube-api-access-j88ls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:03.211058 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.211006    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4a3a584-03a3-4d4e-a034-b70f6b640109-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4a3a584-03a3-4d4e-a034-b70f6b640109" (UID: "c4a3a584-03a3-4d4e-a034-b70f6b640109"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:24:03.212776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.212754    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a3a584-03a3-4d4e-a034-b70f6b640109-kube-api-access-cgs9h" (OuterVolumeSpecName: "kube-api-access-cgs9h") pod "c4a3a584-03a3-4d4e-a034-b70f6b640109" (UID: "c4a3a584-03a3-4d4e-a034-b70f6b640109"). InnerVolumeSpecName "kube-api-access-cgs9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:24:03.311846 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.311762    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4a3a584-03a3-4d4e-a034-b70f6b640109-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:03.311846 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.311789    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgs9h\" (UniqueName: \"kubernetes.io/projected/c4a3a584-03a3-4d4e-a034-b70f6b640109-kube-api-access-cgs9h\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:03.788071 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.788040    2556 generic.go:356] "Generic (PLEG): container finished" podID="0b6069cd-5736-4b83-89fd-f4b0cea430ca" containerID="008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f" exitCode=0
Sep 16 13:24:03.788275 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.788084    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"
Sep 16 13:24:03.788275 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.788123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b" event={"ID":"0b6069cd-5736-4b83-89fd-f4b0cea430ca","Type":"ContainerDied","Data":"008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f"}
Sep 16 13:24:03.788275 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.788172    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b" event={"ID":"0b6069cd-5736-4b83-89fd-f4b0cea430ca","Type":"ContainerDied","Data":"6460412f692fd35a89ce863be748454005351d91f150bd0d5e079396641eb3f2"}
Sep 16 13:24:03.788275 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.788203    2556 scope.go:117] "RemoveContainer" containerID="008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f"
Sep 16 13:24:03.789536 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.789512    2556 generic.go:356] "Generic (PLEG): container finished" podID="c4a3a584-03a3-4d4e-a034-b70f6b640109" containerID="34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395" exitCode=0
Sep 16 13:24:03.789649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.789542    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"
Sep 16 13:24:03.789649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.789553    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t" event={"ID":"c4a3a584-03a3-4d4e-a034-b70f6b640109","Type":"ContainerDied","Data":"34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395"}
Sep 16 13:24:03.789649 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.789612    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t" event={"ID":"c4a3a584-03a3-4d4e-a034-b70f6b640109","Type":"ContainerDied","Data":"116483a7f99cab6c74cc89f2bfc6e5d8d0b5521dbd29084bdd307525b0f6c9eb"}
Sep 16 13:24:03.797195 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.797002    2556 scope.go:117] "RemoveContainer" containerID="008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f"
Sep 16 13:24:03.797322 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:24:03.797304    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f\": container with ID starting with 008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f not found: ID does not exist" containerID="008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f"
Sep 16 13:24:03.797375 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.797334    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f"} err="failed to get container status \"008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f\": rpc error: code = NotFound desc = could not find container \"008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f\": container with ID starting with 008426a5c09c1564a8e5357e1d694a2c69987a34206547232a6b304f13c73a1f not found: ID does not exist"
Sep 16 13:24:03.797375 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.797356    2556 scope.go:117] "RemoveContainer" containerID="34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395"
Sep 16 13:24:03.812755 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.812739    2556 scope.go:117] "RemoveContainer" containerID="34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395"
Sep 16 13:24:03.813026 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:24:03.813009    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395\": container with ID starting with 34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395 not found: ID does not exist" containerID="34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395"
Sep 16 13:24:03.813074 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.813033    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395"} err="failed to get container status \"34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395\": rpc error: code = NotFound desc = could not find container \"34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395\": container with ID starting with 34658bd89e943c7c720e030a98c4f907a1be6d732f5020154f6eee5f79d7a395 not found: ID does not exist"
Sep 16 13:24:03.825815 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.825797    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"]
Sep 16 13:24:03.839390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.839371    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9615f1d6-56f8ddf574-ljr9b"]
Sep 16 13:24:03.860791 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.860771    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"]
Sep 16 13:24:03.865085 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:03.865064    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9615f1d6-55c7d6b89d-nmf6t"]
Sep 16 13:24:04.806641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:04.806577    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b6069cd-5736-4b83-89fd-f4b0cea430ca" path="/var/lib/kubelet/pods/0b6069cd-5736-4b83-89fd-f4b0cea430ca/volumes"
Sep 16 13:24:04.807023 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:04.806987    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4a3a584-03a3-4d4e-a034-b70f6b640109" path="/var/lib/kubelet/pods/c4a3a584-03a3-4d4e-a034-b70f6b640109/volumes"
Sep 16 13:24:05.161051 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:05.161016    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:24:17.310471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.310429    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww" podUID="b659bb4c-5550-49f0-9370-8b71400c9167" containerName="main" containerID="cri-o://3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f" gracePeriod=45
Sep 16 13:24:17.564920 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.564870    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:24:17.630646 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630611    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-model-cache\") pod \"b659bb4c-5550-49f0-9370-8b71400c9167\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") "
Sep 16 13:24:17.630646 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630642    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-home\") pod \"b659bb4c-5550-49f0-9370-8b71400c9167\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") "
Sep 16 13:24:17.630869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630683    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b659bb4c-5550-49f0-9370-8b71400c9167-tls-certs\") pod \"b659bb4c-5550-49f0-9370-8b71400c9167\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") "
Sep 16 13:24:17.630869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630718    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-tmp-dir\") pod \"b659bb4c-5550-49f0-9370-8b71400c9167\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") "
Sep 16 13:24:17.630869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630796    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tt5w\" (UniqueName: \"kubernetes.io/projected/b659bb4c-5550-49f0-9370-8b71400c9167-kube-api-access-9tt5w\") pod \"b659bb4c-5550-49f0-9370-8b71400c9167\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") "
Sep 16 13:24:17.630869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630831    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-dshm\") pod \"b659bb4c-5550-49f0-9370-8b71400c9167\" (UID: \"b659bb4c-5550-49f0-9370-8b71400c9167\") "
Sep 16 13:24:17.631134 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630907    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-model-cache" (OuterVolumeSpecName: "model-cache") pod "b659bb4c-5550-49f0-9370-8b71400c9167" (UID: "b659bb4c-5550-49f0-9370-8b71400c9167"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:24:17.631134 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.630921    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-home" (OuterVolumeSpecName: "home") pod "b659bb4c-5550-49f0-9370-8b71400c9167" (UID: "b659bb4c-5550-49f0-9370-8b71400c9167"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:24:17.631134 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.631033    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b659bb4c-5550-49f0-9370-8b71400c9167" (UID: "b659bb4c-5550-49f0-9370-8b71400c9167"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:24:17.631299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.631156    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:17.631299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.631178    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:17.631299 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.631194    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:17.632974 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.632950    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-dshm" (OuterVolumeSpecName: "dshm") pod "b659bb4c-5550-49f0-9370-8b71400c9167" (UID: "b659bb4c-5550-49f0-9370-8b71400c9167"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:24:17.633098 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.633056    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b659bb4c-5550-49f0-9370-8b71400c9167-kube-api-access-9tt5w" (OuterVolumeSpecName: "kube-api-access-9tt5w") pod "b659bb4c-5550-49f0-9370-8b71400c9167" (UID: "b659bb4c-5550-49f0-9370-8b71400c9167"). InnerVolumeSpecName "kube-api-access-9tt5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:24:17.633098 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.633070    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b659bb4c-5550-49f0-9370-8b71400c9167-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b659bb4c-5550-49f0-9370-8b71400c9167" (UID: "b659bb4c-5550-49f0-9370-8b71400c9167"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:24:17.731927 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.731896    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tt5w\" (UniqueName: \"kubernetes.io/projected/b659bb4c-5550-49f0-9370-8b71400c9167-kube-api-access-9tt5w\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:17.731927 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.731923    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b659bb4c-5550-49f0-9370-8b71400c9167-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:17.731927 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.731934    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b659bb4c-5550-49f0-9370-8b71400c9167-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:24:17.866653 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.866553    2556 generic.go:356] "Generic (PLEG): container finished" podID="b659bb4c-5550-49f0-9370-8b71400c9167" containerID="3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f" exitCode=0
Sep 16 13:24:17.866653 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.866610    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"
Sep 16 13:24:17.866653 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.866633    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww" event={"ID":"b659bb4c-5550-49f0-9370-8b71400c9167","Type":"ContainerDied","Data":"3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f"}
Sep 16 13:24:17.866857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.866680    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww" event={"ID":"b659bb4c-5550-49f0-9370-8b71400c9167","Type":"ContainerDied","Data":"7bbe26251296eb4ca8000708349f1688a536100a64bdaf2f829a5ce1663a1464"}
Sep 16 13:24:17.866857 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.866698    2556 scope.go:117] "RemoveContainer" containerID="3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f"
Sep 16 13:24:17.876147 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.876133    2556 scope.go:117] "RemoveContainer" containerID="3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f"
Sep 16 13:24:17.876441 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:24:17.876424    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f\": container with ID starting with 3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f not found: ID does not exist" containerID="3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f"
Sep 16 13:24:17.876504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.876447    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f"} err="failed to get container status \"3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f\": rpc error: code = NotFound desc = could not find container \"3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f\": container with ID starting with 3f39a53006126377d01f0947c27a5714ce2ee17d44b476e7104a4e761925892f not found: ID does not exist"
Sep 16 13:24:17.905910 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.905887    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"]
Sep 16 13:24:17.915146 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:17.915125    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-15da7b/autores-model-15da7b-kserve-684bcdc8dc-8zpww"]
Sep 16 13:24:18.767034 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:18.766999    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:24:18.799628 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:18.799582    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b659bb4c-5550-49f0-9370-8b71400c9167" path="/var/lib/kubelet/pods/b659bb4c-5550-49f0-9370-8b71400c9167/volumes"
Sep 16 13:24:28.071935 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.071902    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"]
Sep 16 13:24:28.072292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072230    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6069cd-5736-4b83-89fd-f4b0cea430ca" containerName="maas-api"
Sep 16 13:24:28.072292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072241    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6069cd-5736-4b83-89fd-f4b0cea430ca" containerName="maas-api"
Sep 16 13:24:28.072292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072256    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b659bb4c-5550-49f0-9370-8b71400c9167" containerName="main"
Sep 16 13:24:28.072292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072261    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b659bb4c-5550-49f0-9370-8b71400c9167" containerName="main"
Sep 16 13:24:28.072292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072272    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4a3a584-03a3-4d4e-a034-b70f6b640109" containerName="payload-pre-processing"
Sep 16 13:24:28.072292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072277    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4a3a584-03a3-4d4e-a034-b70f6b640109" containerName="payload-pre-processing"
Sep 16 13:24:28.072482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072332    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b659bb4c-5550-49f0-9370-8b71400c9167" containerName="main"
Sep 16 13:24:28.072482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072341    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4a3a584-03a3-4d4e-a034-b70f6b640109" containerName="payload-pre-processing"
Sep 16 13:24:28.072482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.072349    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6069cd-5736-4b83-89fd-f4b0cea430ca" containerName="maas-api"
Sep 16 13:24:28.076742 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.076721    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.081409 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.081387    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f27ea8\"/\"premium-w4-f27ea8-kserve-self-signed-certs\""
Sep 16 13:24:28.137124 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.137097    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"]
Sep 16 13:24:28.221486 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.221453    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-home\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.221486 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.221490    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-dshm\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.221722 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.221512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-model-cache\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.221722 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.221586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-tmp-dir\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.221722 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.221679    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5be301a-1253-4cac-ad1a-03bcd64739e0-tls-certs\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.221836 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.221724    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2dth\" (UniqueName: \"kubernetes.io/projected/b5be301a-1253-4cac-ad1a-03bcd64739e0-kube-api-access-d2dth\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.322662 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.322541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-home\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.322662 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.322582    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-dshm\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.322662 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.322622    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-model-cache\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.322662 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.322656    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-tmp-dir\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.322962 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.322706    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5be301a-1253-4cac-ad1a-03bcd64739e0-tls-certs\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.322962 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.322755    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2dth\" (UniqueName: \"kubernetes.io/projected/b5be301a-1253-4cac-ad1a-03bcd64739e0-kube-api-access-d2dth\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.323071 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.323057    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-home\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.323126 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.323093    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-model-cache\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.323172 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.323147    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-tmp-dir\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.324956 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.324924    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-dshm\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.325206 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.325189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5be301a-1253-4cac-ad1a-03bcd64739e0-tls-certs\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.336848 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.336788    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2dth\" (UniqueName: \"kubernetes.io/projected/b5be301a-1253-4cac-ad1a-03bcd64739e0-kube-api-access-d2dth\") pod \"premium-w4-f27ea8-kserve-9ff744995-dqldp\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") " pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.387114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.387081    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:28.671130 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.671103    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"]
Sep 16 13:24:28.671844 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:24:28.671818    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5be301a_1253_4cac_ad1a_03bcd64739e0.slice/crio-226e158c854c0076f4994b81cb55e234522e5e070afde45ae6fa22615ae20c85 WatchSource:0}: Error finding container 226e158c854c0076f4994b81cb55e234522e5e070afde45ae6fa22615ae20c85: Status 404 returned error can't find the container with id 226e158c854c0076f4994b81cb55e234522e5e070afde45ae6fa22615ae20c85
Sep 16 13:24:28.918576 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.918539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp" event={"ID":"b5be301a-1253-4cac-ad1a-03bcd64739e0","Type":"ContainerStarted","Data":"c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4"}
Sep 16 13:24:28.918757 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.918585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp" event={"ID":"b5be301a-1253-4cac-ad1a-03bcd64739e0","Type":"ContainerStarted","Data":"226e158c854c0076f4994b81cb55e234522e5e070afde45ae6fa22615ae20c85"}
Sep 16 13:24:28.951424 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:28.951328    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp" podStartSLOduration=0.951313363 podStartE2EDuration="951.313363ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:24:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:24:28.944685702 +0000 UTC m=+976.621400155" watchObservedRunningTime="2026-09-16 13:24:28.951313363 +0000 UTC m=+976.628027798"
Sep 16 13:24:35.937442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:35.937365    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:24:38.387451 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:38.387403    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:38.387871 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:38.387472    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:38.399899 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:38.399876    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:38.976090 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:38.976042    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:24:58.127785 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:24:58.127753    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:25:06.519253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.519223    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"]
Sep 16 13:25:06.522879 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.522862    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.525082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.525062    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a21a40\"/\"premium-w0-a21a40-kserve-self-signed-certs\""
Sep 16 13:25:06.541160 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.541128    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"]
Sep 16 13:25:06.636497 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.636468    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h9rt\" (UniqueName: \"kubernetes.io/projected/032179e5-4542-4dcd-bf47-c96596f251d8-kube-api-access-5h9rt\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.636646 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.636508    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-model-cache\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.636646 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.636531    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-home\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.636748 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.636648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-dshm\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.636748 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.636675    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-tmp-dir\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.636748 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.636697    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032179e5-4542-4dcd-bf47-c96596f251d8-tls-certs\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.737592 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.737564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-dshm\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.737713 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.737618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-tmp-dir\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.737713 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.737650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032179e5-4542-4dcd-bf47-c96596f251d8-tls-certs\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.737713 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.737707    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h9rt\" (UniqueName: \"kubernetes.io/projected/032179e5-4542-4dcd-bf47-c96596f251d8-kube-api-access-5h9rt\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.737861 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.737751    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-model-cache\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.737861 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.737785    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-home\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.738099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.738068    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-tmp-dir\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.738209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.738099    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-model-cache\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.738209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.738164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-home\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.739848 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.739823    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-dshm\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.740007 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.739991    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032179e5-4542-4dcd-bf47-c96596f251d8-tls-certs\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.750373 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.750349    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h9rt\" (UniqueName: \"kubernetes.io/projected/032179e5-4542-4dcd-bf47-c96596f251d8-kube-api-access-5h9rt\") pod \"premium-w0-a21a40-kserve-74f46d8d6c-8t8fg\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") " pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:06.832250 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:06.832198    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:07.113398 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:07.113201    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"]
Sep 16 13:25:07.116180 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:25:07.116154    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod032179e5_4542_4dcd_bf47_c96596f251d8.slice/crio-c5e1eee2782f10e6be076050de0a93bc50507d29040eea713ca43b229da3c7e2 WatchSource:0}: Error finding container c5e1eee2782f10e6be076050de0a93bc50507d29040eea713ca43b229da3c7e2: Status 404 returned error can't find the container with id c5e1eee2782f10e6be076050de0a93bc50507d29040eea713ca43b229da3c7e2
Sep 16 13:25:08.102702 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:08.102658    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg" event={"ID":"032179e5-4542-4dcd-bf47-c96596f251d8","Type":"ContainerStarted","Data":"910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf"}
Sep 16 13:25:08.102702 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:08.102703    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg" event={"ID":"032179e5-4542-4dcd-bf47-c96596f251d8","Type":"ContainerStarted","Data":"c5e1eee2782f10e6be076050de0a93bc50507d29040eea713ca43b229da3c7e2"}
Sep 16 13:25:08.134889 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:08.134839    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg" podStartSLOduration=2.13482348 podStartE2EDuration="2.13482348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:25:08.133554718 +0000 UTC m=+1015.810269153" watchObservedRunningTime="2026-09-16 13:25:08.13482348 +0000 UTC m=+1015.811537916"
Sep 16 13:25:16.833187 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:16.833158    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:16.833573 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:16.833206    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:16.845785 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:16.845758    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:17.156394 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:17.156364    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:25:19.580861 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.580825    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"]
Sep 16 13:25:19.585370 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.585353    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.633543 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.633516    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c71ad7\"/\"openshift-service-ca.crt\""
Sep 16 13:25:19.633682 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.633547    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c71ad7\"/\"test-model-c71ad7-kserve-self-signed-certs\""
Sep 16 13:25:19.633682 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.633654    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c71ad7\"/\"default-dockercfg-5s8rf\""
Sep 16 13:25:19.637150 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.637107    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c71ad7\"/\"kube-root-ca.crt\""
Sep 16 13:25:19.641991 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.641969    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-dshm\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.642091 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.642013    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-tmp-dir\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.642091 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.642037    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pc5v\" (UniqueName: \"kubernetes.io/projected/e949880b-26e8-4292-bf87-176fc21b0a97-kube-api-access-4pc5v\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.642091 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.642079    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e949880b-26e8-4292-bf87-176fc21b0a97-tls-certs\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.642214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.642107    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-model-cache\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.642214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.642160    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-home\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.663271 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.663250    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"]
Sep 16 13:25:19.742757 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.742737    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-dshm\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.742860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.742769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-tmp-dir\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.742860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.742788    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pc5v\" (UniqueName: \"kubernetes.io/projected/e949880b-26e8-4292-bf87-176fc21b0a97-kube-api-access-4pc5v\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.742860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.742804    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e949880b-26e8-4292-bf87-176fc21b0a97-tls-certs\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.742860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.742826    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-model-cache\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.742860 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.742860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-home\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.743178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.743155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-tmp-dir\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.743251 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.743194    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-home\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.743251 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.743164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-model-cache\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.745074 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.745055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-dshm\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.745396 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.745380    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e949880b-26e8-4292-bf87-176fc21b0a97-tls-certs\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.754699 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.754679    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pc5v\" (UniqueName: \"kubernetes.io/projected/e949880b-26e8-4292-bf87-176fc21b0a97-kube-api-access-4pc5v\") pod \"test-model-c71ad7-kserve-f5ff98b7f-mmq22\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") " pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:19.894727 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:19.894702    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:20.217979 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:20.217804    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"]
Sep 16 13:25:20.220212 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:25:20.220178    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode949880b_26e8_4292_bf87_176fc21b0a97.slice/crio-114d52890f876313fe8e17f35310bad3f7f10579e4b48d928df0915db5511838 WatchSource:0}: Error finding container 114d52890f876313fe8e17f35310bad3f7f10579e4b48d928df0915db5511838: Status 404 returned error can't find the container with id 114d52890f876313fe8e17f35310bad3f7f10579e4b48d928df0915db5511838
Sep 16 13:25:21.018210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:21.018176    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:25:21.166968 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:21.166939    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22" event={"ID":"e949880b-26e8-4292-bf87-176fc21b0a97","Type":"ContainerStarted","Data":"5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4"}
Sep 16 13:25:21.167102 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:21.166975    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22" event={"ID":"e949880b-26e8-4292-bf87-176fc21b0a97","Type":"ContainerStarted","Data":"114d52890f876313fe8e17f35310bad3f7f10579e4b48d928df0915db5511838"}
Sep 16 13:25:21.201407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:21.201364    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22" podStartSLOduration=2.201350578 podStartE2EDuration="2.201350578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:25:21.197890582 +0000 UTC m=+1028.874605018" watchObservedRunningTime="2026-09-16 13:25:21.201350578 +0000 UTC m=+1028.878065039"
Sep 16 13:25:29.895083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:29.895047    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:29.895083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:29.895087    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:29.907339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:29.907315    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:30.220391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:30.220302    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:25:44.353393 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:25:44.353361    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:26:08.349570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:26:08.349479    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:26:39.454951 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:26:39.454921    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:27:00.548377 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:00.548337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:27:12.707686 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:12.707653    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"]
Sep 16 13:27:27.726398 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:27.726362    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz" podUID="181a10db-31ec-46ae-8359-b29d99fbdace" containerName="main" containerID="cri-o://ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3" gracePeriod=45
Sep 16 13:27:27.970786 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:27.970763    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:27:28.130446 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130421    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-model-cache\") pod \"181a10db-31ec-46ae-8359-b29d99fbdace\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") "
Sep 16 13:27:28.130638 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130462    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/181a10db-31ec-46ae-8359-b29d99fbdace-tls-certs\") pod \"181a10db-31ec-46ae-8359-b29d99fbdace\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") "
Sep 16 13:27:28.130638 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130522    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-home\") pod \"181a10db-31ec-46ae-8359-b29d99fbdace\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") "
Sep 16 13:27:28.130638 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130569    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-dshm\") pod \"181a10db-31ec-46ae-8359-b29d99fbdace\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") "
Sep 16 13:27:28.130638 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130586    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjnk6\" (UniqueName: \"kubernetes.io/projected/181a10db-31ec-46ae-8359-b29d99fbdace-kube-api-access-wjnk6\") pod \"181a10db-31ec-46ae-8359-b29d99fbdace\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") "
Sep 16 13:27:28.130886 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130641    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-tmp-dir\") pod \"181a10db-31ec-46ae-8359-b29d99fbdace\" (UID: \"181a10db-31ec-46ae-8359-b29d99fbdace\") "
Sep 16 13:27:28.130886 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130727    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-model-cache" (OuterVolumeSpecName: "model-cache") pod "181a10db-31ec-46ae-8359-b29d99fbdace" (UID: "181a10db-31ec-46ae-8359-b29d99fbdace"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:27:28.130886 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130839    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-home" (OuterVolumeSpecName: "home") pod "181a10db-31ec-46ae-8359-b29d99fbdace" (UID: "181a10db-31ec-46ae-8359-b29d99fbdace"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:27:28.131083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130962    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:27:28.131083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130976    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:27:28.131083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.130976    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "181a10db-31ec-46ae-8359-b29d99fbdace" (UID: "181a10db-31ec-46ae-8359-b29d99fbdace"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:27:28.132979 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.132949    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-dshm" (OuterVolumeSpecName: "dshm") pod "181a10db-31ec-46ae-8359-b29d99fbdace" (UID: "181a10db-31ec-46ae-8359-b29d99fbdace"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:27:28.132979 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.132953    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181a10db-31ec-46ae-8359-b29d99fbdace-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "181a10db-31ec-46ae-8359-b29d99fbdace" (UID: "181a10db-31ec-46ae-8359-b29d99fbdace"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:27:28.133123 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.132993    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181a10db-31ec-46ae-8359-b29d99fbdace-kube-api-access-wjnk6" (OuterVolumeSpecName: "kube-api-access-wjnk6") pod "181a10db-31ec-46ae-8359-b29d99fbdace" (UID: "181a10db-31ec-46ae-8359-b29d99fbdace"). InnerVolumeSpecName "kube-api-access-wjnk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:27:28.232380 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.232359    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:27:28.232380 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.232378    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/181a10db-31ec-46ae-8359-b29d99fbdace-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:27:28.232552 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.232387    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/181a10db-31ec-46ae-8359-b29d99fbdace-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:27:28.232552 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.232395    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjnk6\" (UniqueName: \"kubernetes.io/projected/181a10db-31ec-46ae-8359-b29d99fbdace-kube-api-access-wjnk6\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:27:28.776260 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.776222    2556 generic.go:356] "Generic (PLEG): container finished" podID="181a10db-31ec-46ae-8359-b29d99fbdace" containerID="ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3" exitCode=0
Sep 16 13:27:28.776689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.776262    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"
Sep 16 13:27:28.776689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.776303    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz" event={"ID":"181a10db-31ec-46ae-8359-b29d99fbdace","Type":"ContainerDied","Data":"ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3"}
Sep 16 13:27:28.776689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.776342    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz" event={"ID":"181a10db-31ec-46ae-8359-b29d99fbdace","Type":"ContainerDied","Data":"86ee347ee2feccf40ad52d3ee1e13f760c85c937ee521bb86483ac32bebafc62"}
Sep 16 13:27:28.776689 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.776361    2556 scope.go:117] "RemoveContainer" containerID="ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3"
Sep 16 13:27:28.785561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.785543    2556 scope.go:117] "RemoveContainer" containerID="ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3"
Sep 16 13:27:28.785831 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:27:28.785811    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3\": container with ID starting with ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3 not found: ID does not exist" containerID="ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3"
Sep 16 13:27:28.785906 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.785845    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3"} err="failed to get container status \"ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3\": rpc error: code = NotFound desc = could not find container \"ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3\": container with ID starting with ebefdececc4498b826f513b4460843326aa9061bca56bf3c1b17cf256f77f7e3 not found: ID does not exist"
Sep 16 13:27:28.807201 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.807179    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"]
Sep 16 13:27:28.812788 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:28.812763    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-48adf5/test-model-48adf5-kserve-6c78f9bdd5-nb2vz"]
Sep 16 13:27:30.748495 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:30.748458    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:27:30.800416 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:30.800385    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="181a10db-31ec-46ae-8359-b29d99fbdace" path="/var/lib/kubelet/pods/181a10db-31ec-46ae-8359-b29d99fbdace/volumes"
Sep 16 13:27:48.359234 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:48.359160    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:27:49.211862 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:27:49.211831    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"]
Sep 16 13:28:04.230790 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.230723    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22" podUID="e949880b-26e8-4292-bf87-176fc21b0a97" containerName="main" containerID="cri-o://5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4" gracePeriod=45
Sep 16 13:28:04.478884 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.478856    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:28:04.521485 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521414    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pc5v\" (UniqueName: \"kubernetes.io/projected/e949880b-26e8-4292-bf87-176fc21b0a97-kube-api-access-4pc5v\") pod \"e949880b-26e8-4292-bf87-176fc21b0a97\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") "
Sep 16 13:28:04.521485 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521461    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-home\") pod \"e949880b-26e8-4292-bf87-176fc21b0a97\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") "
Sep 16 13:28:04.521698 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521506    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e949880b-26e8-4292-bf87-176fc21b0a97-tls-certs\") pod \"e949880b-26e8-4292-bf87-176fc21b0a97\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") "
Sep 16 13:28:04.521698 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521553    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-dshm\") pod \"e949880b-26e8-4292-bf87-176fc21b0a97\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") "
Sep 16 13:28:04.521698 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521640    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-model-cache\") pod \"e949880b-26e8-4292-bf87-176fc21b0a97\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") "
Sep 16 13:28:04.521877 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521702    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-tmp-dir\") pod \"e949880b-26e8-4292-bf87-176fc21b0a97\" (UID: \"e949880b-26e8-4292-bf87-176fc21b0a97\") "
Sep 16 13:28:04.521986 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521936    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-model-cache" (OuterVolumeSpecName: "model-cache") pod "e949880b-26e8-4292-bf87-176fc21b0a97" (UID: "e949880b-26e8-4292-bf87-176fc21b0a97"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:28:04.522070 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.522042    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e949880b-26e8-4292-bf87-176fc21b0a97" (UID: "e949880b-26e8-4292-bf87-176fc21b0a97"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:28:04.522070 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.522061    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:28:04.522161 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.521791    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-home" (OuterVolumeSpecName: "home") pod "e949880b-26e8-4292-bf87-176fc21b0a97" (UID: "e949880b-26e8-4292-bf87-176fc21b0a97"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:28:04.523727 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.523695    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e949880b-26e8-4292-bf87-176fc21b0a97-kube-api-access-4pc5v" (OuterVolumeSpecName: "kube-api-access-4pc5v") pod "e949880b-26e8-4292-bf87-176fc21b0a97" (UID: "e949880b-26e8-4292-bf87-176fc21b0a97"). InnerVolumeSpecName "kube-api-access-4pc5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:28:04.524090 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.524064    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-dshm" (OuterVolumeSpecName: "dshm") pod "e949880b-26e8-4292-bf87-176fc21b0a97" (UID: "e949880b-26e8-4292-bf87-176fc21b0a97"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:28:04.524203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.524084    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e949880b-26e8-4292-bf87-176fc21b0a97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e949880b-26e8-4292-bf87-176fc21b0a97" (UID: "e949880b-26e8-4292-bf87-176fc21b0a97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:28:04.622826 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.622787    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:28:04.622826 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.622822    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pc5v\" (UniqueName: \"kubernetes.io/projected/e949880b-26e8-4292-bf87-176fc21b0a97-kube-api-access-4pc5v\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:28:04.622826 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.622833    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:28:04.623030 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.622842    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e949880b-26e8-4292-bf87-176fc21b0a97-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:28:04.623030 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.622850    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e949880b-26e8-4292-bf87-176fc21b0a97-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:28:04.959204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.959169    2556 generic.go:356] "Generic (PLEG): container finished" podID="e949880b-26e8-4292-bf87-176fc21b0a97" containerID="5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4" exitCode=0
Sep 16 13:28:04.959204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.959209    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"
Sep 16 13:28:04.959413 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.959263    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22" event={"ID":"e949880b-26e8-4292-bf87-176fc21b0a97","Type":"ContainerDied","Data":"5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4"}
Sep 16 13:28:04.959413 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.959304    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22" event={"ID":"e949880b-26e8-4292-bf87-176fc21b0a97","Type":"ContainerDied","Data":"114d52890f876313fe8e17f35310bad3f7f10579e4b48d928df0915db5511838"}
Sep 16 13:28:04.959413 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.959322    2556 scope.go:117] "RemoveContainer" containerID="5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4"
Sep 16 13:28:04.967725 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.967702    2556 scope.go:117] "RemoveContainer" containerID="5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4"
Sep 16 13:28:04.967993 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:28:04.967972    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4\": container with ID starting with 5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4 not found: ID does not exist" containerID="5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4"
Sep 16 13:28:04.968063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.968004    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4"} err="failed to get container status \"5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4\": rpc error: code = NotFound desc = could not find container \"5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4\": container with ID starting with 5fb7b10b0a876d22b4a40677450f6bed61f092e6f01b798f809915a08d5e3be4 not found: ID does not exist"
Sep 16 13:28:04.975025 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.975000    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"]
Sep 16 13:28:04.977026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:04.977007    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c71ad7/test-model-c71ad7-kserve-f5ff98b7f-mmq22"]
Sep 16 13:28:06.800665 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:06.800632    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e949880b-26e8-4292-bf87-176fc21b0a97" path="/var/lib/kubelet/pods/e949880b-26e8-4292-bf87-176fc21b0a97/volumes"
Sep 16 13:28:07.544905 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:07.544857    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:28:12.834365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:12.834335    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:28:12.837940 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:12.837873    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:28:26.940069 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:28:26.940035    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:29:39.686429 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686392    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"]
Sep 16 13:29:39.686959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686739    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181a10db-31ec-46ae-8359-b29d99fbdace" containerName="main"
Sep 16 13:29:39.686959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686751    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="181a10db-31ec-46ae-8359-b29d99fbdace" containerName="main"
Sep 16 13:29:39.686959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686768    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e949880b-26e8-4292-bf87-176fc21b0a97" containerName="main"
Sep 16 13:29:39.686959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686774    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e949880b-26e8-4292-bf87-176fc21b0a97" containerName="main"
Sep 16 13:29:39.686959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686853    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="181a10db-31ec-46ae-8359-b29d99fbdace" containerName="main"
Sep 16 13:29:39.686959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.686865    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e949880b-26e8-4292-bf87-176fc21b0a97" containerName="main"
Sep 16 13:29:39.690567 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.690550    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:39.693044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.693020    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b97e79df\""
Sep 16 13:29:39.693044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.693039    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b97e79df-dockercfg-zfxtw\""
Sep 16 13:29:39.701099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.701078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"]
Sep 16 13:29:39.803375 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.803351    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w44l5\" (UniqueName: \"kubernetes.io/projected/fa5a212f-8dec-4651-ae6d-db5991086d43-kube-api-access-w44l5\") pod \"payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") " pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:39.803487 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.803384    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5a212f-8dec-4651-ae6d-db5991086d43-plugins-config-volume\") pod \"payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") " pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:39.904289 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.904259    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w44l5\" (UniqueName: \"kubernetes.io/projected/fa5a212f-8dec-4651-ae6d-db5991086d43-kube-api-access-w44l5\") pod \"payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") " pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:39.904428 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.904300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5a212f-8dec-4651-ae6d-db5991086d43-plugins-config-volume\") pod \"payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") " pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:39.905037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.905016    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5a212f-8dec-4651-ae6d-db5991086d43-plugins-config-volume\") pod \"payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") " pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:39.912416 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:39.912397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w44l5\" (UniqueName: \"kubernetes.io/projected/fa5a212f-8dec-4651-ae6d-db5991086d43-kube-api-access-w44l5\") pod \"payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") " pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:40.001505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:40.001438    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:40.274413 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:40.274373    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"]
Sep 16 13:29:40.276708 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:29:40.276670    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5a212f_8dec_4651_ae6d_db5991086d43.slice/crio-ad38a2216645a3aa38e1d742004d23c68b20cd8464f2c22618938b0e844d92d9 WatchSource:0}: Error finding container ad38a2216645a3aa38e1d742004d23c68b20cd8464f2c22618938b0e844d92d9: Status 404 returned error can't find the container with id ad38a2216645a3aa38e1d742004d23c68b20cd8464f2c22618938b0e844d92d9
Sep 16 13:29:40.413604 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:40.413564    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5" event={"ID":"fa5a212f-8dec-4651-ae6d-db5991086d43","Type":"ContainerStarted","Data":"8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178"}
Sep 16 13:29:40.413760 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:40.413660    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5" event={"ID":"fa5a212f-8dec-4651-ae6d-db5991086d43","Type":"ContainerStarted","Data":"ad38a2216645a3aa38e1d742004d23c68b20cd8464f2c22618938b0e844d92d9"}
Sep 16 13:29:40.413760 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:40.413696    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:29:40.431213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:40.431170    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5" podStartSLOduration=1.431154916 podStartE2EDuration="1.431154916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:29:40.429652146 +0000 UTC m=+1288.106366581" watchObservedRunningTime="2026-09-16 13:29:40.431154916 +0000 UTC m=+1288.107869394"
Sep 16 13:29:49.053020 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:49.052978    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:29:51.419900 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:29:51.419871    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:30:00.148840 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.148811    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"]
Sep 16 13:30:00.152412 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.152380    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"]
Sep 16 13:30:00.152561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.152510    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:00.154865 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.154844    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nvp5m\""
Sep 16 13:30:00.157132 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.157097    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:00.170630 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.170584    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"]
Sep 16 13:30:00.172789 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.172768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx8nq\" (UniqueName: \"kubernetes.io/projected/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a-kube-api-access-lx8nq\") pod \"maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg\" (UID: \"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:00.173042 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.173015    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7d6b\" (UniqueName: \"kubernetes.io/projected/cbbcb059-b8cc-4cc5-a21f-f2810d051af3-kube-api-access-x7d6b\") pod \"maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56\" (UID: \"cbbcb059-b8cc-4cc5-a21f-f2810d051af3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:00.173751 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.173728    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"]
Sep 16 13:30:00.274133 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.274102    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx8nq\" (UniqueName: \"kubernetes.io/projected/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a-kube-api-access-lx8nq\") pod \"maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg\" (UID: \"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:00.274291 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.274148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7d6b\" (UniqueName: \"kubernetes.io/projected/cbbcb059-b8cc-4cc5-a21f-f2810d051af3-kube-api-access-x7d6b\") pod \"maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56\" (UID: \"cbbcb059-b8cc-4cc5-a21f-f2810d051af3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:00.282844 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.282814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx8nq\" (UniqueName: \"kubernetes.io/projected/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a-kube-api-access-lx8nq\") pod \"maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg\" (UID: \"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:00.282961 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.282814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7d6b\" (UniqueName: \"kubernetes.io/projected/cbbcb059-b8cc-4cc5-a21f-f2810d051af3-kube-api-access-x7d6b\") pod \"maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56\" (UID: \"cbbcb059-b8cc-4cc5-a21f-f2810d051af3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:00.465083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.465001    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:00.474200 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.474179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:00.740715 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.740692    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"]
Sep 16 13:30:00.743062 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:30:00.743035    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd89d7fa_4d2a_4b8a_b545_d9218fe6a10a.slice/crio-9a29d83d81ed56f38b1109852bae15a252cdac06fcc59fff77bcd7bc058e3efb WatchSource:0}: Error finding container 9a29d83d81ed56f38b1109852bae15a252cdac06fcc59fff77bcd7bc058e3efb: Status 404 returned error can't find the container with id 9a29d83d81ed56f38b1109852bae15a252cdac06fcc59fff77bcd7bc058e3efb
Sep 16 13:30:00.745542 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.745524    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:30:00.746400 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:00.746373    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"]
Sep 16 13:30:00.748855 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:30:00.748831    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbcb059_b8cc_4cc5_a21f_f2810d051af3.slice/crio-bcfb8c7de7fbf3406c1e1f3819a2c13bf81c43ae96a883d4635d58018267697b WatchSource:0}: Error finding container bcfb8c7de7fbf3406c1e1f3819a2c13bf81c43ae96a883d4635d58018267697b: Status 404 returned error can't find the container with id bcfb8c7de7fbf3406c1e1f3819a2c13bf81c43ae96a883d4635d58018267697b
Sep 16 13:30:01.517955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:01.517918    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56" event={"ID":"cbbcb059-b8cc-4cc5-a21f-f2810d051af3","Type":"ContainerStarted","Data":"bcfb8c7de7fbf3406c1e1f3819a2c13bf81c43ae96a883d4635d58018267697b"}
Sep 16 13:30:01.519277 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:01.519247    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg" event={"ID":"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a","Type":"ContainerStarted","Data":"9a29d83d81ed56f38b1109852bae15a252cdac06fcc59fff77bcd7bc058e3efb"}
Sep 16 13:30:04.539781 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:04.539747    2556 generic.go:356] "Generic (PLEG): container finished" podID="cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a" containerID="56b17ec88b1f6f290a49f3039bd875a157b5bd22b4271182f00052da65f980c4" exitCode=0
Sep 16 13:30:04.540166 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:04.539839    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg" event={"ID":"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a","Type":"ContainerDied","Data":"56b17ec88b1f6f290a49f3039bd875a157b5bd22b4271182f00052da65f980c4"}
Sep 16 13:30:04.541526 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:04.541504    2556 generic.go:356] "Generic (PLEG): container finished" podID="cbbcb059-b8cc-4cc5-a21f-f2810d051af3" containerID="fed52f01c7e841572a76dfea3de40d96f08abc398060bc789efc48daecb58d94" exitCode=0
Sep 16 13:30:04.541640 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:04.541564    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56" event={"ID":"cbbcb059-b8cc-4cc5-a21f-f2810d051af3","Type":"ContainerDied","Data":"fed52f01c7e841572a76dfea3de40d96f08abc398060bc789efc48daecb58d94"}
Sep 16 13:30:05.672139 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.672111    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:05.710688 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.710664    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:05.721104 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.721082    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx8nq\" (UniqueName: \"kubernetes.io/projected/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a-kube-api-access-lx8nq\") pod \"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a\" (UID: \"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a\") "
Sep 16 13:30:05.723110 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.723085    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a-kube-api-access-lx8nq" (OuterVolumeSpecName: "kube-api-access-lx8nq") pod "cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a" (UID: "cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a"). InnerVolumeSpecName "kube-api-access-lx8nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:30:05.822568 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.822484    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7d6b\" (UniqueName: \"kubernetes.io/projected/cbbcb059-b8cc-4cc5-a21f-f2810d051af3-kube-api-access-x7d6b\") pod \"cbbcb059-b8cc-4cc5-a21f-f2810d051af3\" (UID: \"cbbcb059-b8cc-4cc5-a21f-f2810d051af3\") "
Sep 16 13:30:05.822751 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.822738    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx8nq\" (UniqueName: \"kubernetes.io/projected/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a-kube-api-access-lx8nq\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:30:05.824503 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.824481    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbbcb059-b8cc-4cc5-a21f-f2810d051af3-kube-api-access-x7d6b" (OuterVolumeSpecName: "kube-api-access-x7d6b") pod "cbbcb059-b8cc-4cc5-a21f-f2810d051af3" (UID: "cbbcb059-b8cc-4cc5-a21f-f2810d051af3"). InnerVolumeSpecName "kube-api-access-x7d6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:30:05.923978 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:05.923946    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7d6b\" (UniqueName: \"kubernetes.io/projected/cbbcb059-b8cc-4cc5-a21f-f2810d051af3-kube-api-access-x7d6b\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:30:06.552825 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:06.552793    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56"
Sep 16 13:30:06.552998 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:06.552829    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56" event={"ID":"cbbcb059-b8cc-4cc5-a21f-f2810d051af3","Type":"ContainerDied","Data":"bcfb8c7de7fbf3406c1e1f3819a2c13bf81c43ae96a883d4635d58018267697b"}
Sep 16 13:30:06.552998 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:06.552885    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcfb8c7de7fbf3406c1e1f3819a2c13bf81c43ae96a883d4635d58018267697b"
Sep 16 13:30:06.554117 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:06.554093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg" event={"ID":"cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a","Type":"ContainerDied","Data":"9a29d83d81ed56f38b1109852bae15a252cdac06fcc59fff77bcd7bc058e3efb"}
Sep 16 13:30:06.554117 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:06.554108    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"
Sep 16 13:30:06.554287 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:06.554141    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a29d83d81ed56f38b1109852bae15a252cdac06fcc59fff77bcd7bc058e3efb"
Sep 16 13:30:19.295326 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.295294    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"]
Sep 16 13:30:19.300854 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.300825    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b97e79df-29826090-tqrgg"]
Sep 16 13:30:19.333619 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.333579    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"]
Sep 16 13:30:19.334065 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.333960    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5" podUID="fa5a212f-8dec-4651-ae6d-db5991086d43" containerName="payload-processing" containerID="cri-o://8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178" gracePeriod=30
Sep 16 13:30:19.597506 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.597481    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:30:19.616782 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.616750    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa5a212f-8dec-4651-ae6d-db5991086d43" containerID="8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178" exitCode=0
Sep 16 13:30:19.616907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.616790    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"
Sep 16 13:30:19.616907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.616834    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5" event={"ID":"fa5a212f-8dec-4651-ae6d-db5991086d43","Type":"ContainerDied","Data":"8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178"}
Sep 16 13:30:19.616907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.616884    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5" event={"ID":"fa5a212f-8dec-4651-ae6d-db5991086d43","Type":"ContainerDied","Data":"ad38a2216645a3aa38e1d742004d23c68b20cd8464f2c22618938b0e844d92d9"}
Sep 16 13:30:19.616907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.616906    2556 scope.go:117] "RemoveContainer" containerID="8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178"
Sep 16 13:30:19.626656 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.626639    2556 scope.go:117] "RemoveContainer" containerID="8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178"
Sep 16 13:30:19.626943 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:30:19.626923    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178\": container with ID starting with 8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178 not found: ID does not exist" containerID="8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178"
Sep 16 13:30:19.627014 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.626955    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178"} err="failed to get container status \"8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178\": rpc error: code = NotFound desc = could not find container \"8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178\": container with ID starting with 8ff2672e46d8192a0bd81a0d36c850c8949cff83f8555046685e4c3f6a3ad178 not found: ID does not exist"
Sep 16 13:30:19.741871 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.741838    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5a212f-8dec-4651-ae6d-db5991086d43-plugins-config-volume\") pod \"fa5a212f-8dec-4651-ae6d-db5991086d43\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") "
Sep 16 13:30:19.741871 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.741876    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w44l5\" (UniqueName: \"kubernetes.io/projected/fa5a212f-8dec-4651-ae6d-db5991086d43-kube-api-access-w44l5\") pod \"fa5a212f-8dec-4651-ae6d-db5991086d43\" (UID: \"fa5a212f-8dec-4651-ae6d-db5991086d43\") "
Sep 16 13:30:19.742263 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.742238    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5a212f-8dec-4651-ae6d-db5991086d43-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa5a212f-8dec-4651-ae6d-db5991086d43" (UID: "fa5a212f-8dec-4651-ae6d-db5991086d43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:30:19.743918 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.743897    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5a212f-8dec-4651-ae6d-db5991086d43-kube-api-access-w44l5" (OuterVolumeSpecName: "kube-api-access-w44l5") pod "fa5a212f-8dec-4651-ae6d-db5991086d43" (UID: "fa5a212f-8dec-4651-ae6d-db5991086d43"). InnerVolumeSpecName "kube-api-access-w44l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:30:19.843407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.843311    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5a212f-8dec-4651-ae6d-db5991086d43-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:30:19.843407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.843353    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w44l5\" (UniqueName: \"kubernetes.io/projected/fa5a212f-8dec-4651-ae6d-db5991086d43-kube-api-access-w44l5\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:30:19.941313 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.941279    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"]
Sep 16 13:30:19.948751 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:19.948726    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97e79df-84d6cb6c49-vhsl5"]
Sep 16 13:30:20.800126 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:20.800088    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a" path="/var/lib/kubelet/pods/cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a/volumes"
Sep 16 13:30:20.800504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:20.800461    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5a212f-8dec-4651-ae6d-db5991086d43" path="/var/lib/kubelet/pods/fa5a212f-8dec-4651-ae6d-db5991086d43/volumes"
Sep 16 13:30:26.250577 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:26.250540    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:30:45.957578 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957504    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"]
Sep 16 13:30:45.957939 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957913    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a" containerName="cleanup"
Sep 16 13:30:45.957939 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957927    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a" containerName="cleanup"
Sep 16 13:30:45.958011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957943    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbbcb059-b8cc-4cc5-a21f-f2810d051af3" containerName="cleanup"
Sep 16 13:30:45.958011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957948    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbbcb059-b8cc-4cc5-a21f-f2810d051af3" containerName="cleanup"
Sep 16 13:30:45.958011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957964    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5a212f-8dec-4651-ae6d-db5991086d43" containerName="payload-processing"
Sep 16 13:30:45.958011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.957969    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5a212f-8dec-4651-ae6d-db5991086d43" containerName="payload-processing"
Sep 16 13:30:45.958140 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.958028    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa5a212f-8dec-4651-ae6d-db5991086d43" containerName="payload-processing"
Sep 16 13:30:45.958140 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.958036    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd89d7fa-4d2a-4b8a-b545-d9218fe6a10a" containerName="cleanup"
Sep 16 13:30:45.958140 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.958042    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbbcb059-b8cc-4cc5-a21f-f2810d051af3" containerName="cleanup"
Sep 16 13:30:45.960827 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.960810    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:45.963060 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.963043    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-b0514e\""
Sep 16 13:30:45.976818 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:45.976787    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"]
Sep 16 13:30:46.069261 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.069233    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfgqw\" (UniqueName: \"kubernetes.io/projected/01d67d28-55ed-443c-9aab-43476531e2d7-kube-api-access-sfgqw\") pod \"payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.069381 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.069266    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d67d28-55ed-443c-9aab-43476531e2d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.169801 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.169769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfgqw\" (UniqueName: \"kubernetes.io/projected/01d67d28-55ed-443c-9aab-43476531e2d7-kube-api-access-sfgqw\") pod \"payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.169963 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.169816    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d67d28-55ed-443c-9aab-43476531e2d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.170493 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.170473    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d67d28-55ed-443c-9aab-43476531e2d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.178508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.178486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfgqw\" (UniqueName: \"kubernetes.io/projected/01d67d28-55ed-443c-9aab-43476531e2d7-kube-api-access-sfgqw\") pod \"payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.270368 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.270281    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.551097 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.551075    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"]
Sep 16 13:30:46.553155 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:30:46.553127    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d67d28_55ed_443c_9aab_43476531e2d7.slice/crio-75b497d322482e700f5e32fc252c1f74f0d59461ca10f5cd7f142b2d7c96e524 WatchSource:0}: Error finding container 75b497d322482e700f5e32fc252c1f74f0d59461ca10f5cd7f142b2d7c96e524: Status 404 returned error can't find the container with id 75b497d322482e700f5e32fc252c1f74f0d59461ca10f5cd7f142b2d7c96e524
Sep 16 13:30:46.744229 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.744186    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7" event={"ID":"01d67d28-55ed-443c-9aab-43476531e2d7","Type":"ContainerStarted","Data":"98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173"}
Sep 16 13:30:46.744391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.744252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7" event={"ID":"01d67d28-55ed-443c-9aab-43476531e2d7","Type":"ContainerStarted","Data":"75b497d322482e700f5e32fc252c1f74f0d59461ca10f5cd7f142b2d7c96e524"}
Sep 16 13:30:46.744391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.744374    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:30:46.766612 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:46.766549    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7" podStartSLOduration=1.766534439 podStartE2EDuration="1.766534439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:30:46.764896837 +0000 UTC m=+1354.441611272" watchObservedRunningTime="2026-09-16 13:30:46.766534439 +0000 UTC m=+1354.443248874"
Sep 16 13:30:50.747858 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:50.747822    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:30:57.751284 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:30:57.751243    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:31:16.821231 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.821197    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"]
Sep 16 13:31:16.825877 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.825859    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:16.828361 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.828338    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-7841d9\""
Sep 16 13:31:16.828361 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.828343    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-7841d9-dockercfg-svxrk\""
Sep 16 13:31:16.838052 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.838028    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"]
Sep 16 13:31:16.912795 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.912773    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66fc331d-fd62-474d-8427-215860f0a583-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:16.912908 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:16.912836    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5xwq\" (UniqueName: \"kubernetes.io/projected/66fc331d-fd62-474d-8427-215860f0a583-kube-api-access-c5xwq\") pod \"payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.013179 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.013152    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5xwq\" (UniqueName: \"kubernetes.io/projected/66fc331d-fd62-474d-8427-215860f0a583-kube-api-access-c5xwq\") pod \"payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.013285 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.013243    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66fc331d-fd62-474d-8427-215860f0a583-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.013934 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.013917    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66fc331d-fd62-474d-8427-215860f0a583-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.024692 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.024664    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5xwq\" (UniqueName: \"kubernetes.io/projected/66fc331d-fd62-474d-8427-215860f0a583-kube-api-access-c5xwq\") pod \"payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.136190 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.136164    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.414068 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.414047    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"]
Sep 16 13:31:17.416279 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:31:17.416244    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66fc331d_fd62_474d_8427_215860f0a583.slice/crio-1f7a070655641fe4b8e96620b917f52190001a499b4461afc91aeca5b958fd51 WatchSource:0}: Error finding container 1f7a070655641fe4b8e96620b917f52190001a499b4461afc91aeca5b958fd51: Status 404 returned error can't find the container with id 1f7a070655641fe4b8e96620b917f52190001a499b4461afc91aeca5b958fd51
Sep 16 13:31:17.900125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.900090    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl" event={"ID":"66fc331d-fd62-474d-8427-215860f0a583","Type":"ContainerStarted","Data":"dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6"}
Sep 16 13:31:17.900503 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.900141    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl" event={"ID":"66fc331d-fd62-474d-8427-215860f0a583","Type":"ContainerStarted","Data":"1f7a070655641fe4b8e96620b917f52190001a499b4461afc91aeca5b958fd51"}
Sep 16 13:31:17.900503 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.900221    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:17.920285 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:17.920241    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl" podStartSLOduration=1.920229105 podStartE2EDuration="1.920229105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:31:17.919140272 +0000 UTC m=+1385.595854711" watchObservedRunningTime="2026-09-16 13:31:17.920229105 +0000 UTC m=+1385.596943539"
Sep 16 13:31:27.662759 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:27.662724    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:31:28.907421 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:28.907391    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:31:42.385155 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.385117    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7479-86df02"]
Sep 16 13:31:42.388422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.388405    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:42.390656 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.390626    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mxtqf\""
Sep 16 13:31:42.394101 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.394076    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-86df02"]
Sep 16 13:31:42.519056 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.519023    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6grpm\" (UniqueName: \"kubernetes.io/projected/777b878a-f387-4a29-a526-32cf808f7110-kube-api-access-6grpm\") pod \"test-curl-7479-86df02\" (UID: \"777b878a-f387-4a29-a526-32cf808f7110\") " pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:42.620365 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.620331    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grpm\" (UniqueName: \"kubernetes.io/projected/777b878a-f387-4a29-a526-32cf808f7110-kube-api-access-6grpm\") pod \"test-curl-7479-86df02\" (UID: \"777b878a-f387-4a29-a526-32cf808f7110\") " pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:42.628463 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.628439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grpm\" (UniqueName: \"kubernetes.io/projected/777b878a-f387-4a29-a526-32cf808f7110-kube-api-access-6grpm\") pod \"test-curl-7479-86df02\" (UID: \"777b878a-f387-4a29-a526-32cf808f7110\") " pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:42.699711 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.699658    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:42.971151 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:42.971102    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-86df02"]
Sep 16 13:31:42.973344 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:31:42.973315    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod777b878a_f387_4a29_a526_32cf808f7110.slice/crio-102f47820afa1a5e8a4196da4c0a88e2363763c9199df304250207baa9779b82 WatchSource:0}: Error finding container 102f47820afa1a5e8a4196da4c0a88e2363763c9199df304250207baa9779b82: Status 404 returned error can't find the container with id 102f47820afa1a5e8a4196da4c0a88e2363763c9199df304250207baa9779b82
Sep 16 13:31:43.023471 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:43.023448    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-86df02" event={"ID":"777b878a-f387-4a29-a526-32cf808f7110","Type":"ContainerStarted","Data":"102f47820afa1a5e8a4196da4c0a88e2363763c9199df304250207baa9779b82"}
Sep 16 13:31:44.029737 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.029705    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-86df02" event={"ID":"777b878a-f387-4a29-a526-32cf808f7110","Type":"ContainerStarted","Data":"7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc"}
Sep 16 13:31:44.046716 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.046667    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7479-86df02" podStartSLOduration=2.046650977 podStartE2EDuration="2.046650977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:31:44.045916127 +0000 UTC m=+1411.722630567" watchObservedRunningTime="2026-09-16 13:31:44.046650977 +0000 UTC m=+1411.723365459"
Sep 16 13:31:44.662074 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.662048    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7479-86df02"]
Sep 16 13:31:44.667235 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.667212    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7479-86df02"]
Sep 16 13:31:44.785239 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.785164    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7479-2d453c"]
Sep 16 13:31:44.785515 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.785503    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777b878a-f387-4a29-a526-32cf808f7110" containerName="test-curl-7479-86df02"
Sep 16 13:31:44.785559 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.785517    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="777b878a-f387-4a29-a526-32cf808f7110" containerName="test-curl-7479-86df02"
Sep 16 13:31:44.785628 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.785584    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="777b878a-f387-4a29-a526-32cf808f7110" containerName="test-curl-7479-86df02"
Sep 16 13:31:44.788547 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.788528    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:44.802833 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.802809    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-2d453c"]
Sep 16 13:31:44.939246 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:44.939218    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zffvd\" (UniqueName: \"kubernetes.io/projected/6385768e-02ab-4e8b-9b8a-5e4067457d61-kube-api-access-zffvd\") pod \"test-curl-7479-2d453c\" (UID: \"6385768e-02ab-4e8b-9b8a-5e4067457d61\") " pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:45.033983 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:45.033951    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7479-86df02" container="test-curl-7479-86df02"
Sep 16 13:31:45.040288 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:45.040244    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zffvd\" (UniqueName: \"kubernetes.io/projected/6385768e-02ab-4e8b-9b8a-5e4067457d61-kube-api-access-zffvd\") pod \"test-curl-7479-2d453c\" (UID: \"6385768e-02ab-4e8b-9b8a-5e4067457d61\") " pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:45.049050 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:45.049024    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zffvd\" (UniqueName: \"kubernetes.io/projected/6385768e-02ab-4e8b-9b8a-5e4067457d61-kube-api-access-zffvd\") pod \"test-curl-7479-2d453c\" (UID: \"6385768e-02ab-4e8b-9b8a-5e4067457d61\") " pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:45.099914 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:45.099884    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:45.366312 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:45.366240    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-2d453c"]
Sep 16 13:31:45.369214 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:31:45.369187    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6385768e_02ab_4e8b_9b8a_5e4067457d61.slice/crio-40b442444cdcf785643c63338b3e36a0827c793e682bc4e2b6217c5c65976c8a WatchSource:0}: Error finding container 40b442444cdcf785643c63338b3e36a0827c793e682bc4e2b6217c5c65976c8a: Status 404 returned error can't find the container with id 40b442444cdcf785643c63338b3e36a0827c793e682bc4e2b6217c5c65976c8a
Sep 16 13:31:46.040315 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.040275    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-2d453c" event={"ID":"6385768e-02ab-4e8b-9b8a-5e4067457d61","Type":"ContainerStarted","Data":"e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc"}
Sep 16 13:31:46.040723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.040326    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-2d453c" event={"ID":"6385768e-02ab-4e8b-9b8a-5e4067457d61","Type":"ContainerStarted","Data":"40b442444cdcf785643c63338b3e36a0827c793e682bc4e2b6217c5c65976c8a"}
Sep 16 13:31:46.040723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.040430    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7479-86df02" podUID="777b878a-f387-4a29-a526-32cf808f7110" containerName="test-curl-7479-86df02" containerID="cri-o://7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc" gracePeriod=2
Sep 16 13:31:46.057097 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.057054    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7479-2d453c" podStartSLOduration=2.057040612 podStartE2EDuration="2.057040612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:31:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:31:46.055532295 +0000 UTC m=+1413.732246731" watchObservedRunningTime="2026-09-16 13:31:46.057040612 +0000 UTC m=+1413.733755046"
Sep 16 13:31:46.690566 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.690490    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7479-2d453c"]
Sep 16 13:31:46.694553 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.694527    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7479-2d453c"]
Sep 16 13:31:46.809548 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.809521    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7479-c47b3d"]
Sep 16 13:31:46.809912 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.809897    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6385768e-02ab-4e8b-9b8a-5e4067457d61" containerName="test-curl-7479-2d453c"
Sep 16 13:31:46.809959 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.809914    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6385768e-02ab-4e8b-9b8a-5e4067457d61" containerName="test-curl-7479-2d453c"
Sep 16 13:31:46.809999 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.809986    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6385768e-02ab-4e8b-9b8a-5e4067457d61" containerName="test-curl-7479-2d453c"
Sep 16 13:31:46.813087 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.813072    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:46.816483 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.816459    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-c47b3d"]
Sep 16 13:31:46.854767 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.854744    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ldq9\" (UniqueName: \"kubernetes.io/projected/f8361769-8d1b-4200-99fc-e0033226db8f-kube-api-access-5ldq9\") pod \"test-curl-7479-c47b3d\" (UID: \"f8361769-8d1b-4200-99fc-e0033226db8f\") " pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:46.955222 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.955157    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ldq9\" (UniqueName: \"kubernetes.io/projected/f8361769-8d1b-4200-99fc-e0033226db8f-kube-api-access-5ldq9\") pod \"test-curl-7479-c47b3d\" (UID: \"f8361769-8d1b-4200-99fc-e0033226db8f\") " pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:46.963002 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:46.962980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ldq9\" (UniqueName: \"kubernetes.io/projected/f8361769-8d1b-4200-99fc-e0033226db8f-kube-api-access-5ldq9\") pod \"test-curl-7479-c47b3d\" (UID: \"f8361769-8d1b-4200-99fc-e0033226db8f\") " pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:47.044876 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:47.044849    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7479-2d453c" container="test-curl-7479-2d453c"
Sep 16 13:31:47.122794 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:47.122775    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:47.391711 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:47.391685    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-c47b3d"]
Sep 16 13:31:47.394230 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:31:47.394198    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8361769_8d1b_4200_99fc_e0033226db8f.slice/crio-95f1d4bb73ea4d4528abe25857bb688f6bfb33ad19ce1c544b107e0910a195bf WatchSource:0}: Error finding container 95f1d4bb73ea4d4528abe25857bb688f6bfb33ad19ce1c544b107e0910a195bf: Status 404 returned error can't find the container with id 95f1d4bb73ea4d4528abe25857bb688f6bfb33ad19ce1c544b107e0910a195bf
Sep 16 13:31:48.051659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.051630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-c47b3d" event={"ID":"f8361769-8d1b-4200-99fc-e0033226db8f","Type":"ContainerStarted","Data":"68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf"}
Sep 16 13:31:48.051988 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.051675    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-c47b3d" event={"ID":"f8361769-8d1b-4200-99fc-e0033226db8f","Type":"ContainerStarted","Data":"95f1d4bb73ea4d4528abe25857bb688f6bfb33ad19ce1c544b107e0910a195bf"}
Sep 16 13:31:48.051988 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.051782    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7479-2d453c" podUID="6385768e-02ab-4e8b-9b8a-5e4067457d61" containerName="test-curl-7479-2d453c" containerID="cri-o://e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc" gracePeriod=2
Sep 16 13:31:48.068846 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.068805    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7479-c47b3d" podStartSLOduration=2.068790174 podStartE2EDuration="2.068790174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:31:48.067079731 +0000 UTC m=+1415.743794178" watchObservedRunningTime="2026-09-16 13:31:48.068790174 +0000 UTC m=+1415.745504650"
Sep 16 13:31:48.211114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.211092    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:48.264624 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.264586    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6grpm\" (UniqueName: \"kubernetes.io/projected/777b878a-f387-4a29-a526-32cf808f7110-kube-api-access-6grpm\") pod \"777b878a-f387-4a29-a526-32cf808f7110\" (UID: \"777b878a-f387-4a29-a526-32cf808f7110\") "
Sep 16 13:31:48.266479 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.266447    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777b878a-f387-4a29-a526-32cf808f7110-kube-api-access-6grpm" (OuterVolumeSpecName: "kube-api-access-6grpm") pod "777b878a-f387-4a29-a526-32cf808f7110" (UID: "777b878a-f387-4a29-a526-32cf808f7110"). InnerVolumeSpecName "kube-api-access-6grpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:31:48.365327 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.365279    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6grpm\" (UniqueName: \"kubernetes.io/projected/777b878a-f387-4a29-a526-32cf808f7110-kube-api-access-6grpm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:31:48.650675 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.650648    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7479-c47b3d"]
Sep 16 13:31:48.655466 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.655443    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7479-c47b3d"]
Sep 16 13:31:48.775280 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.775255    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7479-0a4f01"]
Sep 16 13:31:48.775633 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.775618    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8361769-8d1b-4200-99fc-e0033226db8f" containerName="test-curl-7479-c47b3d"
Sep 16 13:31:48.775633 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.775634    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8361769-8d1b-4200-99fc-e0033226db8f" containerName="test-curl-7479-c47b3d"
Sep 16 13:31:48.775723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.775694    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8361769-8d1b-4200-99fc-e0033226db8f" containerName="test-curl-7479-c47b3d"
Sep 16 13:31:48.778650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.778635    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:48.784865 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.784844    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-0a4f01"]
Sep 16 13:31:48.800046 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.800022    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="777b878a-f387-4a29-a526-32cf808f7110" path="/var/lib/kubelet/pods/777b878a-f387-4a29-a526-32cf808f7110/volumes"
Sep 16 13:31:48.869995 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.869976    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkx94\" (UniqueName: \"kubernetes.io/projected/155faed2-b10d-4bce-a9af-3ce08c678bfd-kube-api-access-kkx94\") pod \"test-curl-7479-0a4f01\" (UID: \"155faed2-b10d-4bce-a9af-3ce08c678bfd\") " pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:48.970501 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.970452    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkx94\" (UniqueName: \"kubernetes.io/projected/155faed2-b10d-4bce-a9af-3ce08c678bfd-kube-api-access-kkx94\") pod \"test-curl-7479-0a4f01\" (UID: \"155faed2-b10d-4bce-a9af-3ce08c678bfd\") " pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:48.979862 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:48.979843    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkx94\" (UniqueName: \"kubernetes.io/projected/155faed2-b10d-4bce-a9af-3ce08c678bfd-kube-api-access-kkx94\") pod \"test-curl-7479-0a4f01\" (UID: \"155faed2-b10d-4bce-a9af-3ce08c678bfd\") " pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:49.058213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.058181    2556 generic.go:356] "Generic (PLEG): container finished" podID="777b878a-f387-4a29-a526-32cf808f7110" containerID="7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc" exitCode=137
Sep 16 13:31:49.058586 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.058223    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-86df02"
Sep 16 13:31:49.058586 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.058272    2556 scope.go:117] "RemoveContainer" containerID="7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc"
Sep 16 13:31:49.058586 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:49.058430    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7479-c47b3d" container="test-curl-7479-c47b3d"
Sep 16 13:31:49.066644 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.066621    2556 scope.go:117] "RemoveContainer" containerID="7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc"
Sep 16 13:31:49.066887 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:49.066867    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc\": container with ID starting with 7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc not found: ID does not exist" containerID="7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc"
Sep 16 13:31:49.066943 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.066900    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc"} err="failed to get container status \"7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc\": rpc error: code = NotFound desc = could not find container \"7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc\": container with ID starting with 7670fb0573edbc4d20baa311df64a12d50a762580f122776e46faad95e89e9cc not found: ID does not exist"
Sep 16 13:31:49.089293 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.089266    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:49.358540 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:49.358467    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7479-0a4f01"]
Sep 16 13:31:49.361408 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:31:49.361376    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod155faed2_b10d_4bce_a9af_3ce08c678bfd.slice/crio-48d08f7e08bfaaf6cc0f8432455b0e9c5084b4a0140a263acf039af7b28d2292 WatchSource:0}: Error finding container 48d08f7e08bfaaf6cc0f8432455b0e9c5084b4a0140a263acf039af7b28d2292: Status 404 returned error can't find the container with id 48d08f7e08bfaaf6cc0f8432455b0e9c5084b4a0140a263acf039af7b28d2292
Sep 16 13:31:50.065672 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.065641    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-0a4f01" event={"ID":"155faed2-b10d-4bce-a9af-3ce08c678bfd","Type":"ContainerStarted","Data":"40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f"}
Sep 16 13:31:50.066059 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.065672    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7479-c47b3d" podUID="f8361769-8d1b-4200-99fc-e0033226db8f" containerName="test-curl-7479-c47b3d" containerID="cri-o://68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf" gracePeriod=2
Sep 16 13:31:50.066059 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.065686    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7479-0a4f01" event={"ID":"155faed2-b10d-4bce-a9af-3ce08c678bfd","Type":"ContainerStarted","Data":"48d08f7e08bfaaf6cc0f8432455b0e9c5084b4a0140a263acf039af7b28d2292"}
Sep 16 13:31:50.080963 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.080918    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7479-0a4f01" podStartSLOduration=2.080902186 podStartE2EDuration="2.080902186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:31:50.079884505 +0000 UTC m=+1417.756598941" watchObservedRunningTime="2026-09-16 13:31:50.080902186 +0000 UTC m=+1417.757616624"
Sep 16 13:31:50.228313 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.228294    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:50.280073 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.280052    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zffvd\" (UniqueName: \"kubernetes.io/projected/6385768e-02ab-4e8b-9b8a-5e4067457d61-kube-api-access-zffvd\") pod \"6385768e-02ab-4e8b-9b8a-5e4067457d61\" (UID: \"6385768e-02ab-4e8b-9b8a-5e4067457d61\") "
Sep 16 13:31:50.282043 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.282020    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6385768e-02ab-4e8b-9b8a-5e4067457d61-kube-api-access-zffvd" (OuterVolumeSpecName: "kube-api-access-zffvd") pod "6385768e-02ab-4e8b-9b8a-5e4067457d61" (UID: "6385768e-02ab-4e8b-9b8a-5e4067457d61"). InnerVolumeSpecName "kube-api-access-zffvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:31:50.380589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.380562    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zffvd\" (UniqueName: \"kubernetes.io/projected/6385768e-02ab-4e8b-9b8a-5e4067457d61-kube-api-access-zffvd\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:31:50.702865 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.702794    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7479-0a4f01"]
Sep 16 13:31:50.704887 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.704863    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7479-0a4f01"]
Sep 16 13:31:50.800535 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:50.800507    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6385768e-02ab-4e8b-9b8a-5e4067457d61" path="/var/lib/kubelet/pods/6385768e-02ab-4e8b-9b8a-5e4067457d61/volumes"
Sep 16 13:31:51.072024 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:51.071952    2556 generic.go:356] "Generic (PLEG): container finished" podID="6385768e-02ab-4e8b-9b8a-5e4067457d61" containerID="e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc" exitCode=137
Sep 16 13:31:51.072024 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:51.072015    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-2d453c"
Sep 16 13:31:51.072421 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:51.072056    2556 scope.go:117] "RemoveContainer" containerID="e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc"
Sep 16 13:31:51.072421 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:51.072162    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7479-0a4f01" container="test-curl-7479-0a4f01"
Sep 16 13:31:51.081105 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:51.081087    2556 scope.go:117] "RemoveContainer" containerID="e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc"
Sep 16 13:31:51.081372 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:51.081350    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc\": container with ID starting with e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc not found: ID does not exist" containerID="e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc"
Sep 16 13:31:51.081437 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:51.081380    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc"} err="failed to get container status \"e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc\": rpc error: code = NotFound desc = could not find container \"e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc\": container with ID starting with e3ed66e170580fc3e1152acb5149fdb792716ceac66df313f24feee4fdd00fbc not found: ID does not exist"
Sep 16 13:31:52.078252 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:52.078219    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7479-0a4f01" podUID="155faed2-b10d-4bce-a9af-3ce08c678bfd" containerName="test-curl-7479-0a4f01" containerID="cri-o://40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f" gracePeriod=2
Sep 16 13:31:52.246983 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:52.246961    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:52.296163 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:52.296143    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ldq9\" (UniqueName: \"kubernetes.io/projected/f8361769-8d1b-4200-99fc-e0033226db8f-kube-api-access-5ldq9\") pod \"f8361769-8d1b-4200-99fc-e0033226db8f\" (UID: \"f8361769-8d1b-4200-99fc-e0033226db8f\") "
Sep 16 13:31:52.298039 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:52.298015    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8361769-8d1b-4200-99fc-e0033226db8f-kube-api-access-5ldq9" (OuterVolumeSpecName: "kube-api-access-5ldq9") pod "f8361769-8d1b-4200-99fc-e0033226db8f" (UID: "f8361769-8d1b-4200-99fc-e0033226db8f"). InnerVolumeSpecName "kube-api-access-5ldq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:31:52.397578 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:52.397546    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ldq9\" (UniqueName: \"kubernetes.io/projected/f8361769-8d1b-4200-99fc-e0033226db8f-kube-api-access-5ldq9\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:31:52.799970 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:52.799891    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8361769-8d1b-4200-99fc-e0033226db8f" path="/var/lib/kubelet/pods/f8361769-8d1b-4200-99fc-e0033226db8f/volumes"
Sep 16 13:31:53.084063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:53.083998    2556 generic.go:356] "Generic (PLEG): container finished" podID="f8361769-8d1b-4200-99fc-e0033226db8f" containerID="68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf" exitCode=137
Sep 16 13:31:53.084063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:53.084034    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-c47b3d"
Sep 16 13:31:53.084461 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:53.084095    2556 scope.go:117] "RemoveContainer" containerID="68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf"
Sep 16 13:31:53.092627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:53.092584    2556 scope.go:117] "RemoveContainer" containerID="68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf"
Sep 16 13:31:53.092917 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:53.092898    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf\": container with ID starting with 68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf not found: ID does not exist" containerID="68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf"
Sep 16 13:31:53.092978 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:53.092925    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf"} err="failed to get container status \"68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf\": rpc error: code = NotFound desc = could not find container \"68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf\": container with ID starting with 68faf5f1f0a1e19aa3cbf4db260827e65acaef0cf20b941bb0bd43c38ccc7dcf not found: ID does not exist"
Sep 16 13:31:54.268086 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:54.268059    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:54.311720 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:54.311689    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkx94\" (UniqueName: \"kubernetes.io/projected/155faed2-b10d-4bce-a9af-3ce08c678bfd-kube-api-access-kkx94\") pod \"155faed2-b10d-4bce-a9af-3ce08c678bfd\" (UID: \"155faed2-b10d-4bce-a9af-3ce08c678bfd\") "
Sep 16 13:31:54.313706 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:54.313682    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/155faed2-b10d-4bce-a9af-3ce08c678bfd-kube-api-access-kkx94" (OuterVolumeSpecName: "kube-api-access-kkx94") pod "155faed2-b10d-4bce-a9af-3ce08c678bfd" (UID: "155faed2-b10d-4bce-a9af-3ce08c678bfd"). InnerVolumeSpecName "kube-api-access-kkx94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:31:54.412481 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:54.412458    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkx94\" (UniqueName: \"kubernetes.io/projected/155faed2-b10d-4bce-a9af-3ce08c678bfd-kube-api-access-kkx94\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:31:54.800977 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:54.800898    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="155faed2-b10d-4bce-a9af-3ce08c678bfd" path="/var/lib/kubelet/pods/155faed2-b10d-4bce-a9af-3ce08c678bfd/volumes"
Sep 16 13:31:55.096114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:55.096033    2556 generic.go:356] "Generic (PLEG): container finished" podID="155faed2-b10d-4bce-a9af-3ce08c678bfd" containerID="40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f" exitCode=137
Sep 16 13:31:55.096114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:55.096073    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7479-0a4f01"
Sep 16 13:31:55.096114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:55.096097    2556 scope.go:117] "RemoveContainer" containerID="40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f"
Sep 16 13:31:55.104891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:55.104874    2556 scope.go:117] "RemoveContainer" containerID="40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f"
Sep 16 13:31:55.105143 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:31:55.105121    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f\": container with ID starting with 40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f not found: ID does not exist" containerID="40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f"
Sep 16 13:31:55.105231 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:55.105153    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f"} err="failed to get container status \"40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f\": rpc error: code = NotFound desc = could not find container \"40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f\": container with ID starting with 40cf891dea26ebb0f167723bfea5a0831bee63494727d35da021990f74bdf10f not found: ID does not exist"
Sep 16 13:31:59.639901 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.639866    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"]
Sep 16 13:31:59.640282 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.640217    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="155faed2-b10d-4bce-a9af-3ce08c678bfd" containerName="test-curl-7479-0a4f01"
Sep 16 13:31:59.640282 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.640228    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="155faed2-b10d-4bce-a9af-3ce08c678bfd" containerName="test-curl-7479-0a4f01"
Sep 16 13:31:59.640353 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.640290    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="155faed2-b10d-4bce-a9af-3ce08c678bfd" containerName="test-curl-7479-0a4f01"
Sep 16 13:31:59.644637 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.644617    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.646861 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.646842    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4a235ace-serving-cert\""
Sep 16 13:31:59.657911 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.657884    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"]
Sep 16 13:31:59.763042 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.763012    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x796h\" (UniqueName: \"kubernetes.io/projected/ef1db77e-27c3-4605-84a6-58ff65889c39-kube-api-access-x796h\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.763186 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.763055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-maas-api-tls\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.763186 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.763132    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-metrics-certs\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.763259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.763188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1db77e-27c3-4605-84a6-58ff65889c39-openshift-service-ca\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.864526 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.864499    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-metrics-certs\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.864659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.864533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1db77e-27c3-4605-84a6-58ff65889c39-openshift-service-ca\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.864659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.864574    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x796h\" (UniqueName: \"kubernetes.io/projected/ef1db77e-27c3-4605-84a6-58ff65889c39-kube-api-access-x796h\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.864659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.864616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-maas-api-tls\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.865293 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.865271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1db77e-27c3-4605-84a6-58ff65889c39-openshift-service-ca\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.867008 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.866989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-metrics-certs\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.867093 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.867038    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-maas-api-tls\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.872640 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.872615    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x796h\" (UniqueName: \"kubernetes.io/projected/ef1db77e-27c3-4605-84a6-58ff65889c39-kube-api-access-x796h\") pod \"maas-api-e2e-mt-4a235ace-7585b45b56-86rrc\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:31:59.954261 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:31:59.954200    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:32:00.255379 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.255356    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"]
Sep 16 13:32:00.257026 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:32:00.256997    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1db77e_27c3_4605_84a6_58ff65889c39.slice/crio-34de733ef019464664d6b46edbc73be77aabcf78e9f935377568c4faf2f4b143 WatchSource:0}: Error finding container 34de733ef019464664d6b46edbc73be77aabcf78e9f935377568c4faf2f4b143: Status 404 returned error can't find the container with id 34de733ef019464664d6b46edbc73be77aabcf78e9f935377568c4faf2f4b143
Sep 16 13:32:00.514087 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.514013    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"]
Sep 16 13:32:00.518776 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.518753    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.521084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.521062    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4a235ace-dockercfg-tzb8b\""
Sep 16 13:32:00.521170 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.521141    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4a235ace\""
Sep 16 13:32:00.531986 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.531967    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"]
Sep 16 13:32:00.535080 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.535067    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.539311 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.539290    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"]
Sep 16 13:32:00.549869 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.549846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"]
Sep 16 13:32:00.571783 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.571757    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv64b\" (UniqueName: \"kubernetes.io/projected/79736d5b-5da9-48b9-b739-a225e04035cf-kube-api-access-mv64b\") pod \"payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.571872 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.571793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79736d5b-5da9-48b9-b739-a225e04035cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.571915 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.571869    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-plugins-config-volume\") pod \"payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") " pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.571915 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.571908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cqd6\" (UniqueName: \"kubernetes.io/projected/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-kube-api-access-8cqd6\") pod \"payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") " pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.674285 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.673234    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79736d5b-5da9-48b9-b739-a225e04035cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.674285 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.674204    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79736d5b-5da9-48b9-b739-a225e04035cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.680228 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.674855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-plugins-config-volume\") pod \"payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") " pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.680228 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.674925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqd6\" (UniqueName: \"kubernetes.io/projected/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-kube-api-access-8cqd6\") pod \"payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") " pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.680228 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.675008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv64b\" (UniqueName: \"kubernetes.io/projected/79736d5b-5da9-48b9-b739-a225e04035cf-kube-api-access-mv64b\") pod \"payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.680228 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.676066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-plugins-config-volume\") pod \"payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") " pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.684332 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.684310    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqd6\" (UniqueName: \"kubernetes.io/projected/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-kube-api-access-8cqd6\") pod \"payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") " pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:00.684520 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.684501    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv64b\" (UniqueName: \"kubernetes.io/projected/79736d5b-5da9-48b9-b739-a225e04035cf-kube-api-access-mv64b\") pod \"payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.829422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.829359    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:00.845593 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:00.845572    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:01.109103 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.109076    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"]
Sep 16 13:32:01.111623 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:32:01.111577    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79736d5b_5da9_48b9_b739_a225e04035cf.slice/crio-531f4491933d6b9bab05abca5b9b61137f9753afd3ed6fbebe6f640869bcdd9d WatchSource:0}: Error finding container 531f4491933d6b9bab05abca5b9b61137f9753afd3ed6fbebe6f640869bcdd9d: Status 404 returned error can't find the container with id 531f4491933d6b9bab05abca5b9b61137f9753afd3ed6fbebe6f640869bcdd9d
Sep 16 13:32:01.128856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.128835    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"]
Sep 16 13:32:01.130962 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:32:01.130936    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795f6a37_ba3b_4f23_9009_b93c7b4f40eb.slice/crio-703dfc8cdd961fb33971ab028acf5bb47ea5631f6f6bce17f2bd7d39081533dc WatchSource:0}: Error finding container 703dfc8cdd961fb33971ab028acf5bb47ea5631f6f6bce17f2bd7d39081533dc: Status 404 returned error can't find the container with id 703dfc8cdd961fb33971ab028acf5bb47ea5631f6f6bce17f2bd7d39081533dc
Sep 16 13:32:01.131331 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.131308    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc" event={"ID":"ef1db77e-27c3-4605-84a6-58ff65889c39","Type":"ContainerStarted","Data":"b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5"}
Sep 16 13:32:01.131439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.131359    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc" event={"ID":"ef1db77e-27c3-4605-84a6-58ff65889c39","Type":"ContainerStarted","Data":"34de733ef019464664d6b46edbc73be77aabcf78e9f935377568c4faf2f4b143"}
Sep 16 13:32:01.131439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.131391    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:32:01.132771 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.132739    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr" event={"ID":"79736d5b-5da9-48b9-b739-a225e04035cf","Type":"ContainerStarted","Data":"531f4491933d6b9bab05abca5b9b61137f9753afd3ed6fbebe6f640869bcdd9d"}
Sep 16 13:32:01.151312 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:01.151266    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc" podStartSLOduration=1.7488383779999999 podStartE2EDuration="2.151249554s" totalImagesPullingTime="402.411176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:31:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:32:01.150143053 +0000 UTC m=+1428.826857490" watchObservedRunningTime="2026-09-16 13:32:01.151249554 +0000 UTC m=+1428.827963990"
Sep 16 13:32:02.140104 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.140059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b" event={"ID":"795f6a37-ba3b-4f23-9009-b93c7b4f40eb","Type":"ContainerStarted","Data":"36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4"}
Sep 16 13:32:02.140559 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.140213    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:02.140559 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.140236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b" event={"ID":"795f6a37-ba3b-4f23-9009-b93c7b4f40eb","Type":"ContainerStarted","Data":"703dfc8cdd961fb33971ab028acf5bb47ea5631f6f6bce17f2bd7d39081533dc"}
Sep 16 13:32:02.141694 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.141668    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr" event={"ID":"79736d5b-5da9-48b9-b739-a225e04035cf","Type":"ContainerStarted","Data":"1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f"}
Sep 16 13:32:02.141888 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.141872    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:02.157305 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.157256    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b" podStartSLOduration=2.157238089 podStartE2EDuration="2.157238089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:32:02.156574577 +0000 UTC m=+1429.833289011" watchObservedRunningTime="2026-09-16 13:32:02.157238089 +0000 UTC m=+1429.833952523"
Sep 16 13:32:02.172019 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:02.171975    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr" podStartSLOduration=2.171956922 podStartE2EDuration="2.171956922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:32:02.171474567 +0000 UTC m=+1429.848189002" watchObservedRunningTime="2026-09-16 13:32:02.171956922 +0000 UTC m=+1429.848671352"
Sep 16 13:32:07.146874 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:07.146790    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:32:13.148863 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:13.148830    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:32:13.149337 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:13.149132    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:32:13.545747 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:13.545713    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:32:44.660609 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:44.660567    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:32:45.246907 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.246875    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"]
Sep 16 13:32:45.249902 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.249886    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.253099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.253075    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4a235ace\"/\"kube-root-ca.crt\""
Sep 16 13:32:45.254167 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.254145    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4a235ace\"/\"default-dockercfg-w5x8g\""
Sep 16 13:32:45.254270 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.254166    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4a235ace\"/\"openshift-service-ca.crt\""
Sep 16 13:32:45.254270 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.254206    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4a235ace\"/\"e2e-lifecycle-model-4a235ace-kserve-self-signed-certs\""
Sep 16 13:32:45.258567 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.258544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69gm7\" (UniqueName: \"kubernetes.io/projected/6f9bbba8-2263-4760-8106-a38164936770-kube-api-access-69gm7\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.258692 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.258669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-tmp-dir\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.258788 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.258774    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-dshm\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.258849 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.258835    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-model-cache\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.258893 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.258868    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9bbba8-2263-4760-8106-a38164936770-tls-certs\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.258940 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.258897    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-home\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.269405 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.269380    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"]
Sep 16 13:32:45.359739 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.359712    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69gm7\" (UniqueName: \"kubernetes.io/projected/6f9bbba8-2263-4760-8106-a38164936770-kube-api-access-69gm7\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.359856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.359759    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-tmp-dir\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.359856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.359810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-dshm\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.359856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.359833    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-model-cache\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.359856 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.359854    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9bbba8-2263-4760-8106-a38164936770-tls-certs\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.360048 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.359880    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-home\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.360366 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.360330    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-tmp-dir\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.360366 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.360358    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-model-cache\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.360532 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.360385    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-home\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.362100 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.362079    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-dshm\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.362734 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.362704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9bbba8-2263-4760-8106-a38164936770-tls-certs\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.368500 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.368478    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69gm7\" (UniqueName: \"kubernetes.io/projected/6f9bbba8-2263-4760-8106-a38164936770-kube-api-access-69gm7\") pod \"e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") " pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.560175 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.560091    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:45.843523 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:45.843363    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"]
Sep 16 13:32:45.848451 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:32:45.848391    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f9bbba8_2263_4760_8106_a38164936770.slice/crio-c2055f3673bb771cd01459b5fd8782372a82173b1760b57668c29fbfbd43084a WatchSource:0}: Error finding container c2055f3673bb771cd01459b5fd8782372a82173b1760b57668c29fbfbd43084a: Status 404 returned error can't find the container with id c2055f3673bb771cd01459b5fd8782372a82173b1760b57668c29fbfbd43084a
Sep 16 13:32:46.371178 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:46.371141    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9" event={"ID":"6f9bbba8-2263-4760-8106-a38164936770","Type":"ContainerStarted","Data":"722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c"}
Sep 16 13:32:46.371357 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:46.371187    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9" event={"ID":"6f9bbba8-2263-4760-8106-a38164936770","Type":"ContainerStarted","Data":"c2055f3673bb771cd01459b5fd8782372a82173b1760b57668c29fbfbd43084a"}
Sep 16 13:32:46.416578 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:46.416531    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9" podStartSLOduration=1.416515892 podStartE2EDuration="1.416515892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:32:46.415280144 +0000 UTC m=+1474.091994580" watchObservedRunningTime="2026-09-16 13:32:46.416515892 +0000 UTC m=+1474.093230342"
Sep 16 13:32:55.561015 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:55.560972    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:55.561489 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:55.561029    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:55.573541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:55.573511    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:32:56.433968 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:32:56.433942    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:33:05.063278 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:33:05.063244    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:33:12.863571 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:33:12.863541    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:33:12.868196 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:33:12.868174    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:34:45.624728 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:34:45.624695    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"]
Sep 16 13:34:45.629710 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:34:45.629680    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"]
Sep 16 13:35:00.645250 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:00.645180    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg" podUID="032179e5-4542-4dcd-bf47-c96596f251d8" containerName="main" containerID="cri-o://910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf" gracePeriod=45
Sep 16 13:35:00.649011 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:00.648964    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf" podUID="faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" containerName="main" containerID="cri-o://0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c" gracePeriod=45
Sep 16 13:35:00.916291 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:00.916268    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:35:00.919074 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:00.919056    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:35:01.006867 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.006841    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h9rt\" (UniqueName: \"kubernetes.io/projected/032179e5-4542-4dcd-bf47-c96596f251d8-kube-api-access-5h9rt\") pod \"032179e5-4542-4dcd-bf47-c96596f251d8\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") "
Sep 16 13:35:01.007032 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.006887    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-dshm\") pod \"032179e5-4542-4dcd-bf47-c96596f251d8\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") "
Sep 16 13:35:01.007032 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.006942    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-home\") pod \"032179e5-4542-4dcd-bf47-c96596f251d8\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") "
Sep 16 13:35:01.007032 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.006964    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-model-cache\") pod \"032179e5-4542-4dcd-bf47-c96596f251d8\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") "
Sep 16 13:35:01.007032 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007019    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tmp-dir\") pod \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") "
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007042    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-tmp-dir\") pod \"032179e5-4542-4dcd-bf47-c96596f251d8\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") "
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007098    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-home\") pod \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") "
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007127    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-dshm\") pod \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") "
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007154    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tls-certs\") pod \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") "
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007194    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgjbn\" (UniqueName: \"kubernetes.io/projected/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-kube-api-access-mgjbn\") pod \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") "
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007221    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-home" (OuterVolumeSpecName: "home") pod "032179e5-4542-4dcd-bf47-c96596f251d8" (UID: "032179e5-4542-4dcd-bf47-c96596f251d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.007259 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007241    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "032179e5-4542-4dcd-bf47-c96596f251d8" (UID: "032179e5-4542-4dcd-bf47-c96596f251d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.007636 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007232    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032179e5-4542-4dcd-bf47-c96596f251d8-tls-certs\") pod \"032179e5-4542-4dcd-bf47-c96596f251d8\" (UID: \"032179e5-4542-4dcd-bf47-c96596f251d8\") "
Sep 16 13:35:01.007636 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007344    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-model-cache\") pod \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\" (UID: \"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5\") "
Sep 16 13:35:01.007636 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007617    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" (UID: "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.007822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007799    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.007822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007811    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" (UID: "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.007949 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007829    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.007949 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007844    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.008064 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.007984    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "032179e5-4542-4dcd-bf47-c96596f251d8" (UID: "032179e5-4542-4dcd-bf47-c96596f251d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.008064 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.008031    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-home" (OuterVolumeSpecName: "home") pod "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" (UID: "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.009704 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.009675    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" (UID: "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:35:01.009704 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.009686    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/032179e5-4542-4dcd-bf47-c96596f251d8-kube-api-access-5h9rt" (OuterVolumeSpecName: "kube-api-access-5h9rt") pod "032179e5-4542-4dcd-bf47-c96596f251d8" (UID: "032179e5-4542-4dcd-bf47-c96596f251d8"). InnerVolumeSpecName "kube-api-access-5h9rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:35:01.009884 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.009829    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-kube-api-access-mgjbn" (OuterVolumeSpecName: "kube-api-access-mgjbn") pod "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" (UID: "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5"). InnerVolumeSpecName "kube-api-access-mgjbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:35:01.009884 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.009831    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-dshm" (OuterVolumeSpecName: "dshm") pod "032179e5-4542-4dcd-bf47-c96596f251d8" (UID: "032179e5-4542-4dcd-bf47-c96596f251d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.010071 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.010058    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/032179e5-4542-4dcd-bf47-c96596f251d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "032179e5-4542-4dcd-bf47-c96596f251d8" (UID: "032179e5-4542-4dcd-bf47-c96596f251d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:35:01.010071 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.010062    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-dshm" (OuterVolumeSpecName: "dshm") pod "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" (UID: "faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:01.068122 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.068093    2556 generic.go:356] "Generic (PLEG): container finished" podID="032179e5-4542-4dcd-bf47-c96596f251d8" containerID="910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf" exitCode=0
Sep 16 13:35:01.068235 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.068142    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"
Sep 16 13:35:01.068235 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.068183    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg" event={"ID":"032179e5-4542-4dcd-bf47-c96596f251d8","Type":"ContainerDied","Data":"910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf"}
Sep 16 13:35:01.068352 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.068234    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg" event={"ID":"032179e5-4542-4dcd-bf47-c96596f251d8","Type":"ContainerDied","Data":"c5e1eee2782f10e6be076050de0a93bc50507d29040eea713ca43b229da3c7e2"}
Sep 16 13:35:01.068352 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.068257    2556 scope.go:117] "RemoveContainer" containerID="910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf"
Sep 16 13:35:01.069630 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.069582    2556 generic.go:356] "Generic (PLEG): container finished" podID="faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" containerID="0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c" exitCode=0
Sep 16 13:35:01.069630 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.069625    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf" event={"ID":"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5","Type":"ContainerDied","Data":"0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c"}
Sep 16 13:35:01.069824 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.069658    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"
Sep 16 13:35:01.069824 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.069659    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf" event={"ID":"faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5","Type":"ContainerDied","Data":"31acc5cdc0e98c6be0f945dd6002f30fb9a88bea65b4a41834380759fa499209"}
Sep 16 13:35:01.080932 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.080915    2556 scope.go:117] "RemoveContainer" containerID="910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf"
Sep 16 13:35:01.081390 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:35:01.081269    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf\": container with ID starting with 910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf not found: ID does not exist" containerID="910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf"
Sep 16 13:35:01.081390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.081296    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf"} err="failed to get container status \"910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf\": rpc error: code = NotFound desc = could not find container \"910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf\": container with ID starting with 910ecbe1f87dba83cb8524c7cb42841e3ba78163e79a15343d985c80843c3dcf not found: ID does not exist"
Sep 16 13:35:01.081390 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.081313    2556 scope.go:117] "RemoveContainer" containerID="0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c"
Sep 16 13:35:01.089978 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.089960    2556 scope.go:117] "RemoveContainer" containerID="0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c"
Sep 16 13:35:01.090215 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:35:01.090199    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c\": container with ID starting with 0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c not found: ID does not exist" containerID="0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c"
Sep 16 13:35:01.090254 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.090222    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c"} err="failed to get container status \"0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c\": rpc error: code = NotFound desc = could not find container \"0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c\": container with ID starting with 0d76a2398b01a3c22a7e283498cc540d611b57fc8f5c4f0f0dd2eb8c3dfb9c6c not found: ID does not exist"
Sep 16 13:35:01.095991 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.095970    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"]
Sep 16 13:35:01.102911 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.102890    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/premium-w0-a21a40-kserve-74f46d8d6c-8t8fg"]
Sep 16 13:35:01.109188 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109171    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109190    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h9rt\" (UniqueName: \"kubernetes.io/projected/032179e5-4542-4dcd-bf47-c96596f251d8-kube-api-access-5h9rt\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109199    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109209    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032179e5-4542-4dcd-bf47-c96596f251d8-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109217    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109224    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109233    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109243 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109243    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgjbn\" (UniqueName: \"kubernetes.io/projected/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5-kube-api-access-mgjbn\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.109467 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.109251    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032179e5-4542-4dcd-bf47-c96596f251d8-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:35:01.112156 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.112132    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"]
Sep 16 13:35:01.116787 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:01.116767    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a21a40/simulator-w0-a21a40-kserve-6589b5f978-jwzzf"]
Sep 16 13:35:02.800281 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:02.800252    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="032179e5-4542-4dcd-bf47-c96596f251d8" path="/var/lib/kubelet/pods/032179e5-4542-4dcd-bf47-c96596f251d8/volumes"
Sep 16 13:35:02.800680 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:02.800666    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" path="/var/lib/kubelet/pods/faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5/volumes"
Sep 16 13:35:29.446800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:35:29.446765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:36:12.975191 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:12.975160    2556 scope.go:117] "RemoveContainer" containerID="56b17ec88b1f6f290a49f3039bd875a157b5bd22b4271182f00052da65f980c4"
Sep 16 13:36:24.513060 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:24.513024    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"]
Sep 16 13:36:24.539547 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:24.539516    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"]
Sep 16 13:36:36.042277 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.042203    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"]
Sep 16 13:36:36.042724 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.042405    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc" podUID="ef1db77e-27c3-4605-84a6-58ff65889c39" containerName="maas-api" containerID="cri-o://b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5" gracePeriod=30
Sep 16 13:36:36.209109 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.209077    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"]
Sep 16 13:36:36.209498 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.209308    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b" podUID="795f6a37-ba3b-4f23-9009-b93c7b4f40eb" containerName="payload-processing" containerID="cri-o://36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4" gracePeriod=30
Sep 16 13:36:36.247785 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.247750    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"]
Sep 16 13:36:36.248083 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.248027    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr" podUID="79736d5b-5da9-48b9-b739-a225e04035cf" containerName="payload-pre-processing" containerID="cri-o://1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f" gracePeriod=30
Sep 16 13:36:36.367803 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.367780    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:36:36.498081 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.498045    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-maas-api-tls\") pod \"ef1db77e-27c3-4605-84a6-58ff65889c39\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") "
Sep 16 13:36:36.498237 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.498090    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1db77e-27c3-4605-84a6-58ff65889c39-openshift-service-ca\") pod \"ef1db77e-27c3-4605-84a6-58ff65889c39\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") "
Sep 16 13:36:36.498237 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.498131    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-metrics-certs\") pod \"ef1db77e-27c3-4605-84a6-58ff65889c39\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") "
Sep 16 13:36:36.498237 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.498230    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x796h\" (UniqueName: \"kubernetes.io/projected/ef1db77e-27c3-4605-84a6-58ff65889c39-kube-api-access-x796h\") pod \"ef1db77e-27c3-4605-84a6-58ff65889c39\" (UID: \"ef1db77e-27c3-4605-84a6-58ff65889c39\") "
Sep 16 13:36:36.498551 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.498520    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef1db77e-27c3-4605-84a6-58ff65889c39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef1db77e-27c3-4605-84a6-58ff65889c39" (UID: "ef1db77e-27c3-4605-84a6-58ff65889c39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:36:36.501530 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.501480    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ef1db77e-27c3-4605-84a6-58ff65889c39" (UID: "ef1db77e-27c3-4605-84a6-58ff65889c39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:36:36.501675 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.501565    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef1db77e-27c3-4605-84a6-58ff65889c39-kube-api-access-x796h" (OuterVolumeSpecName: "kube-api-access-x796h") pod "ef1db77e-27c3-4605-84a6-58ff65889c39" (UID: "ef1db77e-27c3-4605-84a6-58ff65889c39"). InnerVolumeSpecName "kube-api-access-x796h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:36:36.501675 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.501583    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef1db77e-27c3-4605-84a6-58ff65889c39" (UID: "ef1db77e-27c3-4605-84a6-58ff65889c39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:36:36.516873 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.516854    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:36:36.528245 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.528225    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:36:36.540270 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.540250    2556 generic.go:356] "Generic (PLEG): container finished" podID="79736d5b-5da9-48b9-b739-a225e04035cf" containerID="1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f" exitCode=0
Sep 16 13:36:36.540359 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.540283    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"
Sep 16 13:36:36.540359 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.540337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr" event={"ID":"79736d5b-5da9-48b9-b739-a225e04035cf","Type":"ContainerDied","Data":"1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f"}
Sep 16 13:36:36.540458 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.540385    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr" event={"ID":"79736d5b-5da9-48b9-b739-a225e04035cf","Type":"ContainerDied","Data":"531f4491933d6b9bab05abca5b9b61137f9753afd3ed6fbebe6f640869bcdd9d"}
Sep 16 13:36:36.540458 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.540412    2556 scope.go:117] "RemoveContainer" containerID="1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f"
Sep 16 13:36:36.541820 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.541793    2556 generic.go:356] "Generic (PLEG): container finished" podID="ef1db77e-27c3-4605-84a6-58ff65889c39" containerID="b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5" exitCode=0
Sep 16 13:36:36.541924 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.541838    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"
Sep 16 13:36:36.541924 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.541870    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc" event={"ID":"ef1db77e-27c3-4605-84a6-58ff65889c39","Type":"ContainerDied","Data":"b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5"}
Sep 16 13:36:36.541924 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.541905    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc" event={"ID":"ef1db77e-27c3-4605-84a6-58ff65889c39","Type":"ContainerDied","Data":"34de733ef019464664d6b46edbc73be77aabcf78e9f935377568c4faf2f4b143"}
Sep 16 13:36:36.543200 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.543180    2556 generic.go:356] "Generic (PLEG): container finished" podID="795f6a37-ba3b-4f23-9009-b93c7b4f40eb" containerID="36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4" exitCode=0
Sep 16 13:36:36.543283 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.543230    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"
Sep 16 13:36:36.543338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.543235    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b" event={"ID":"795f6a37-ba3b-4f23-9009-b93c7b4f40eb","Type":"ContainerDied","Data":"36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4"}
Sep 16 13:36:36.543384 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.543364    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b" event={"ID":"795f6a37-ba3b-4f23-9009-b93c7b4f40eb","Type":"ContainerDied","Data":"703dfc8cdd961fb33971ab028acf5bb47ea5631f6f6bce17f2bd7d39081533dc"}
Sep 16 13:36:36.548947 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.548924    2556 scope.go:117] "RemoveContainer" containerID="1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f"
Sep 16 13:36:36.549340 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:36:36.549204    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f\": container with ID starting with 1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f not found: ID does not exist" containerID="1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f"
Sep 16 13:36:36.549340 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.549231    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f"} err="failed to get container status \"1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f\": rpc error: code = NotFound desc = could not find container \"1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f\": container with ID starting with 1f53361a2d1180c9fc71a1354fb6d29a6e3b75a9f8bb6bab04907fd252e56c6f not found: ID does not exist"
Sep 16 13:36:36.549340 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.549248    2556 scope.go:117] "RemoveContainer" containerID="b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5"
Sep 16 13:36:36.556978 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.556964    2556 scope.go:117] "RemoveContainer" containerID="b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5"
Sep 16 13:36:36.557225 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:36:36.557209    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5\": container with ID starting with b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5 not found: ID does not exist" containerID="b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5"
Sep 16 13:36:36.557273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.557233    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5"} err="failed to get container status \"b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5\": rpc error: code = NotFound desc = could not find container \"b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5\": container with ID starting with b414a9a2c6af9b20ea697beaac864635f47e3c5b58375f8587302e97b6117be5 not found: ID does not exist"
Sep 16 13:36:36.557273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.557246    2556 scope.go:117] "RemoveContainer" containerID="36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4"
Sep 16 13:36:36.565956 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.565937    2556 scope.go:117] "RemoveContainer" containerID="36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4"
Sep 16 13:36:36.566388 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:36:36.566364    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4\": container with ID starting with 36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4 not found: ID does not exist" containerID="36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4"
Sep 16 13:36:36.566482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.566399    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4"} err="failed to get container status \"36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4\": rpc error: code = NotFound desc = could not find container \"36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4\": container with ID starting with 36de10b60e2cee19fe9fd64f657606eb584fb59a273bc507c806ad547b8225c4 not found: ID does not exist"
Sep 16 13:36:36.598830 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.598802    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"]
Sep 16 13:36:36.598934 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.598909    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-plugins-config-volume\") pod \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") "
Sep 16 13:36:36.598997 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.598948    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv64b\" (UniqueName: \"kubernetes.io/projected/79736d5b-5da9-48b9-b739-a225e04035cf-kube-api-access-mv64b\") pod \"79736d5b-5da9-48b9-b739-a225e04035cf\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") "
Sep 16 13:36:36.599069 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599052    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79736d5b-5da9-48b9-b739-a225e04035cf-plugins-config-volume\") pod \"79736d5b-5da9-48b9-b739-a225e04035cf\" (UID: \"79736d5b-5da9-48b9-b739-a225e04035cf\") "
Sep 16 13:36:36.599125 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599098    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cqd6\" (UniqueName: \"kubernetes.io/projected/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-kube-api-access-8cqd6\") pod \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\" (UID: \"795f6a37-ba3b-4f23-9009-b93c7b4f40eb\") "
Sep 16 13:36:36.599316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599296    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.599316 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599313    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x796h\" (UniqueName: \"kubernetes.io/projected/ef1db77e-27c3-4605-84a6-58ff65889c39-kube-api-access-x796h\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.599475 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599327    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef1db77e-27c3-4605-84a6-58ff65889c39-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.599475 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599339    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1db77e-27c3-4605-84a6-58ff65889c39-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.599475 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599327    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "795f6a37-ba3b-4f23-9009-b93c7b4f40eb" (UID: "795f6a37-ba3b-4f23-9009-b93c7b4f40eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:36:36.599758 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.599713    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79736d5b-5da9-48b9-b739-a225e04035cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79736d5b-5da9-48b9-b739-a225e04035cf" (UID: "79736d5b-5da9-48b9-b739-a225e04035cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:36:36.601879 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.601859    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-kube-api-access-8cqd6" (OuterVolumeSpecName: "kube-api-access-8cqd6") pod "795f6a37-ba3b-4f23-9009-b93c7b4f40eb" (UID: "795f6a37-ba3b-4f23-9009-b93c7b4f40eb"). InnerVolumeSpecName "kube-api-access-8cqd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:36:36.601963 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.601885    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79736d5b-5da9-48b9-b739-a225e04035cf-kube-api-access-mv64b" (OuterVolumeSpecName: "kube-api-access-mv64b") pod "79736d5b-5da9-48b9-b739-a225e04035cf" (UID: "79736d5b-5da9-48b9-b739-a225e04035cf"). InnerVolumeSpecName "kube-api-access-mv64b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:36:36.602535 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.602515    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4a235ace-7585b45b56-86rrc"]
Sep 16 13:36:36.700122 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.700091    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79736d5b-5da9-48b9-b739-a225e04035cf-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.700122 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.700117    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cqd6\" (UniqueName: \"kubernetes.io/projected/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-kube-api-access-8cqd6\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.700310 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.700131    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6a37-ba3b-4f23-9009-b93c7b4f40eb-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.700310 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.700147    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv64b\" (UniqueName: \"kubernetes.io/projected/79736d5b-5da9-48b9-b739-a225e04035cf-kube-api-access-mv64b\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:36.799768 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.799690    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef1db77e-27c3-4605-84a6-58ff65889c39" path="/var/lib/kubelet/pods/ef1db77e-27c3-4605-84a6-58ff65889c39/volumes"
Sep 16 13:36:36.875580 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.875553    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"]
Sep 16 13:36:36.892496 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.892469    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4a235ace-54765658b6-hj7dr"]
Sep 16 13:36:36.921753 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.921730    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"]
Sep 16 13:36:36.935336 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:36.935307    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a235ace-cdf76899c-jq88b"]
Sep 16 13:36:38.806845 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:38.806803    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="795f6a37-ba3b-4f23-9009-b93c7b4f40eb" path="/var/lib/kubelet/pods/795f6a37-ba3b-4f23-9009-b93c7b4f40eb/volumes"
Sep 16 13:36:38.807329 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:38.807306    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79736d5b-5da9-48b9-b739-a225e04035cf" path="/var/lib/kubelet/pods/79736d5b-5da9-48b9-b739-a225e04035cf/volumes"
Sep 16 13:36:39.531933 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.531897    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp" podUID="b5be301a-1253-4cac-ad1a-03bcd64739e0" containerName="main" containerID="cri-o://c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4" gracePeriod=45
Sep 16 13:36:39.557430 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.557396    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq" podUID="b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" containerName="main" containerID="cri-o://c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e" gracePeriod=45
Sep 16 13:36:39.822590 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.822562    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:36:39.826338 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.826316    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:36:39.928894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.928867    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tls-certs\") pod \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") "
Sep 16 13:36:39.929067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.928922    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-dshm\") pod \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") "
Sep 16 13:36:39.929067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.928946    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-model-cache\") pod \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") "
Sep 16 13:36:39.929067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.928969    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5be301a-1253-4cac-ad1a-03bcd64739e0-tls-certs\") pod \"b5be301a-1253-4cac-ad1a-03bcd64739e0\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") "
Sep 16 13:36:39.929067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.928986    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2dth\" (UniqueName: \"kubernetes.io/projected/b5be301a-1253-4cac-ad1a-03bcd64739e0-kube-api-access-d2dth\") pod \"b5be301a-1253-4cac-ad1a-03bcd64739e0\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") "
Sep 16 13:36:39.929067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tmp-dir\") pod \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") "
Sep 16 13:36:39.929067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929032    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-tmp-dir\") pod \"b5be301a-1253-4cac-ad1a-03bcd64739e0\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") "
Sep 16 13:36:39.929588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929071    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctg5l\" (UniqueName: \"kubernetes.io/projected/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-kube-api-access-ctg5l\") pod \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") "
Sep 16 13:36:39.929588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929098    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-dshm\") pod \"b5be301a-1253-4cac-ad1a-03bcd64739e0\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") "
Sep 16 13:36:39.929588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929173    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-home\") pod \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\" (UID: \"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5\") "
Sep 16 13:36:39.929588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929180    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" (UID: "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.929588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929237    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-model-cache\") pod \"b5be301a-1253-4cac-ad1a-03bcd64739e0\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") "
Sep 16 13:36:39.929588 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929274    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-home\") pod \"b5be301a-1253-4cac-ad1a-03bcd64739e0\" (UID: \"b5be301a-1253-4cac-ad1a-03bcd64739e0\") "
Sep 16 13:36:39.930057 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929650    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-home" (OuterVolumeSpecName: "home") pod "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" (UID: "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.930057 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929690    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-home" (OuterVolumeSpecName: "home") pod "b5be301a-1253-4cac-ad1a-03bcd64739e0" (UID: "b5be301a-1253-4cac-ad1a-03bcd64739e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.930057 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929705    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:39.930057 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.929907    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5be301a-1253-4cac-ad1a-03bcd64739e0" (UID: "b5be301a-1253-4cac-ad1a-03bcd64739e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.930842 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.930810    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5be301a-1253-4cac-ad1a-03bcd64739e0" (UID: "b5be301a-1253-4cac-ad1a-03bcd64739e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.931805 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.931775    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" (UID: "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:36:39.932008 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.931977    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" (UID: "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.932808 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.932637    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5be301a-1253-4cac-ad1a-03bcd64739e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5be301a-1253-4cac-ad1a-03bcd64739e0" (UID: "b5be301a-1253-4cac-ad1a-03bcd64739e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:36:39.932808 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.932703    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5be301a-1253-4cac-ad1a-03bcd64739e0-kube-api-access-d2dth" (OuterVolumeSpecName: "kube-api-access-d2dth") pod "b5be301a-1253-4cac-ad1a-03bcd64739e0" (UID: "b5be301a-1253-4cac-ad1a-03bcd64739e0"). InnerVolumeSpecName "kube-api-access-d2dth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:36:39.932984 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.932811    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-dshm" (OuterVolumeSpecName: "dshm") pod "b5be301a-1253-4cac-ad1a-03bcd64739e0" (UID: "b5be301a-1253-4cac-ad1a-03bcd64739e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.932984 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.932816    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-dshm" (OuterVolumeSpecName: "dshm") pod "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" (UID: "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:39.933413 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:39.933389    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-kube-api-access-ctg5l" (OuterVolumeSpecName: "kube-api-access-ctg5l") pod "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" (UID: "b58f4381-54b8-4ee0-a185-1bec0ef2c2a5"). InnerVolumeSpecName "kube-api-access-ctg5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:36:40.030499 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030476    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030499 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030498    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5be301a-1253-4cac-ad1a-03bcd64739e0-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030509    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2dth\" (UniqueName: \"kubernetes.io/projected/b5be301a-1253-4cac-ad1a-03bcd64739e0-kube-api-access-d2dth\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030518    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030526    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030533    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctg5l\" (UniqueName: \"kubernetes.io/projected/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-kube-api-access-ctg5l\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030541    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030550    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030557    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030565    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5be301a-1253-4cac-ad1a-03bcd64739e0-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.030677 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.030572    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:40.564867 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.564836    2556 generic.go:356] "Generic (PLEG): container finished" podID="b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" containerID="c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e" exitCode=0
Sep 16 13:36:40.565013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.564882    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"
Sep 16 13:36:40.565013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.564916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq" event={"ID":"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5","Type":"ContainerDied","Data":"c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e"}
Sep 16 13:36:40.565013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.564955    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq" event={"ID":"b58f4381-54b8-4ee0-a185-1bec0ef2c2a5","Type":"ContainerDied","Data":"387fb2adf24b0e17b253a66b70672a6133a4e3a615db2723b551ab87e906e1ce"}
Sep 16 13:36:40.565013 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.564972    2556 scope.go:117] "RemoveContainer" containerID="c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e"
Sep 16 13:36:40.566509 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.566484    2556 generic.go:356] "Generic (PLEG): container finished" podID="b5be301a-1253-4cac-ad1a-03bcd64739e0" containerID="c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4" exitCode=0
Sep 16 13:36:40.566626 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.566520    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"
Sep 16 13:36:40.566626 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.566523    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp" event={"ID":"b5be301a-1253-4cac-ad1a-03bcd64739e0","Type":"ContainerDied","Data":"c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4"}
Sep 16 13:36:40.566735 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.566647    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp" event={"ID":"b5be301a-1253-4cac-ad1a-03bcd64739e0","Type":"ContainerDied","Data":"226e158c854c0076f4994b81cb55e234522e5e070afde45ae6fa22615ae20c85"}
Sep 16 13:36:40.573331 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.573308    2556 scope.go:117] "RemoveContainer" containerID="c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e"
Sep 16 13:36:40.573721 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:36:40.573698    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e\": container with ID starting with c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e not found: ID does not exist" containerID="c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e"
Sep 16 13:36:40.573822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.573730    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e"} err="failed to get container status \"c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e\": rpc error: code = NotFound desc = could not find container \"c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e\": container with ID starting with c339cf801faccf25081d4174c77b9e4057cb503d30cc973d931d1ee709b9fb6e not found: ID does not exist"
Sep 16 13:36:40.573822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.573747    2556 scope.go:117] "RemoveContainer" containerID="c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4"
Sep 16 13:36:40.581459 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.581443    2556 scope.go:117] "RemoveContainer" containerID="c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4"
Sep 16 13:36:40.581745 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:36:40.581724    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4\": container with ID starting with c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4 not found: ID does not exist" containerID="c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4"
Sep 16 13:36:40.581828 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.581750    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4"} err="failed to get container status \"c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4\": rpc error: code = NotFound desc = could not find container \"c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4\": container with ID starting with c15aafe922987509f4d8c82cc446f5c0913434e88a53a1bd3451b479645fa7c4 not found: ID does not exist"
Sep 16 13:36:40.596063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.596040    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"]
Sep 16 13:36:40.601532 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.601514    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/simulator-w4-f27ea8-kserve-8f879dc8b-rcfwq"]
Sep 16 13:36:40.615294 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.615275    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"]
Sep 16 13:36:40.618831 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.618813    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f27ea8/premium-w4-f27ea8-kserve-9ff744995-dqldp"]
Sep 16 13:36:40.799550 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.799495    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" path="/var/lib/kubelet/pods/b58f4381-54b8-4ee0-a185-1bec0ef2c2a5/volumes"
Sep 16 13:36:40.799892 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:40.799878    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5be301a-1253-4cac-ad1a-03bcd64739e0" path="/var/lib/kubelet/pods/b5be301a-1253-4cac-ad1a-03bcd64739e0/volumes"
Sep 16 13:36:42.076961 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:42.076918    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"]
Sep 16 13:36:43.651653 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:43.651049    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:36:57.095746 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.095702    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9" podUID="6f9bbba8-2263-4760-8106-a38164936770" containerName="main" containerID="cri-o://722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c" gracePeriod=45
Sep 16 13:36:57.335207 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.335184    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:36:57.470044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470010    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9bbba8-2263-4760-8106-a38164936770-tls-certs\") pod \"6f9bbba8-2263-4760-8106-a38164936770\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") "
Sep 16 13:36:57.470204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470058    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69gm7\" (UniqueName: \"kubernetes.io/projected/6f9bbba8-2263-4760-8106-a38164936770-kube-api-access-69gm7\") pod \"6f9bbba8-2263-4760-8106-a38164936770\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") "
Sep 16 13:36:57.470204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470081    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-model-cache\") pod \"6f9bbba8-2263-4760-8106-a38164936770\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") "
Sep 16 13:36:57.470204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470099    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-home\") pod \"6f9bbba8-2263-4760-8106-a38164936770\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") "
Sep 16 13:36:57.470204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470137    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-dshm\") pod \"6f9bbba8-2263-4760-8106-a38164936770\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") "
Sep 16 13:36:57.470204 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470167    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-tmp-dir\") pod \"6f9bbba8-2263-4760-8106-a38164936770\" (UID: \"6f9bbba8-2263-4760-8106-a38164936770\") "
Sep 16 13:36:57.470479 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470393    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f9bbba8-2263-4760-8106-a38164936770" (UID: "6f9bbba8-2263-4760-8106-a38164936770"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:57.470479 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470415    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-home" (OuterVolumeSpecName: "home") pod "6f9bbba8-2263-4760-8106-a38164936770" (UID: "6f9bbba8-2263-4760-8106-a38164936770"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:57.470561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.470517    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f9bbba8-2263-4760-8106-a38164936770" (UID: "6f9bbba8-2263-4760-8106-a38164936770"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:57.472223 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.472202    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f9bbba8-2263-4760-8106-a38164936770-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f9bbba8-2263-4760-8106-a38164936770" (UID: "6f9bbba8-2263-4760-8106-a38164936770"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:36:57.472290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.472252    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-dshm" (OuterVolumeSpecName: "dshm") pod "6f9bbba8-2263-4760-8106-a38164936770" (UID: "6f9bbba8-2263-4760-8106-a38164936770"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:36:57.472290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.472272    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9bbba8-2263-4760-8106-a38164936770-kube-api-access-69gm7" (OuterVolumeSpecName: "kube-api-access-69gm7") pod "6f9bbba8-2263-4760-8106-a38164936770" (UID: "6f9bbba8-2263-4760-8106-a38164936770"). InnerVolumeSpecName "kube-api-access-69gm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:36:57.570822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.570791    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:57.570822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.570817    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9bbba8-2263-4760-8106-a38164936770-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:57.570822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.570827    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69gm7\" (UniqueName: \"kubernetes.io/projected/6f9bbba8-2263-4760-8106-a38164936770-kube-api-access-69gm7\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:57.571036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.570837    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:57.571036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.570846    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:57.571036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.570853    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9bbba8-2263-4760-8106-a38164936770-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:36:57.653075 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.653046    2556 generic.go:356] "Generic (PLEG): container finished" podID="6f9bbba8-2263-4760-8106-a38164936770" containerID="722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c" exitCode=0
Sep 16 13:36:57.653201 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.653088    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"
Sep 16 13:36:57.653201 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.653093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9" event={"ID":"6f9bbba8-2263-4760-8106-a38164936770","Type":"ContainerDied","Data":"722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c"}
Sep 16 13:36:57.653201 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.653121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9" event={"ID":"6f9bbba8-2263-4760-8106-a38164936770","Type":"ContainerDied","Data":"c2055f3673bb771cd01459b5fd8782372a82173b1760b57668c29fbfbd43084a"}
Sep 16 13:36:57.653201 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.653137    2556 scope.go:117] "RemoveContainer" containerID="722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c"
Sep 16 13:36:57.661930 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.661913    2556 scope.go:117] "RemoveContainer" containerID="722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c"
Sep 16 13:36:57.662212 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:36:57.662194    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c\": container with ID starting with 722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c not found: ID does not exist" containerID="722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c"
Sep 16 13:36:57.662265 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.662222    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c"} err="failed to get container status \"722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c\": rpc error: code = NotFound desc = could not find container \"722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c\": container with ID starting with 722f004b60643185ceff95c11ee7ed408652494efb2a3b62b4acfb6e807a9b7c not found: ID does not exist"
Sep 16 13:36:57.678124 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.678095    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"]
Sep 16 13:36:57.680442 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:57.680420    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4a235ace/e2e-lifecycle-model-4a235ace-kserve-7866597ff-t8nl9"]
Sep 16 13:36:58.800122 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:58.800083    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9bbba8-2263-4760-8106-a38164936770" path="/var/lib/kubelet/pods/6f9bbba8-2263-4760-8106-a38164936770/volumes"
Sep 16 13:36:59.108489 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108417    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"]
Sep 16 13:36:59.108801 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108787    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032179e5-4542-4dcd-bf47-c96596f251d8" containerName="main"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108803    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="032179e5-4542-4dcd-bf47-c96596f251d8" containerName="main"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108810    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9bbba8-2263-4760-8106-a38164936770" containerName="main"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108816    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9bbba8-2263-4760-8106-a38164936770" containerName="main"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108824    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795f6a37-ba3b-4f23-9009-b93c7b4f40eb" containerName="payload-processing"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108830    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="795f6a37-ba3b-4f23-9009-b93c7b4f40eb" containerName="payload-processing"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108838    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" containerName="main"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108844    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" containerName="main"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108851    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79736d5b-5da9-48b9-b739-a225e04035cf" containerName="payload-pre-processing"
Sep 16 13:36:59.108864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108856    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="79736d5b-5da9-48b9-b739-a225e04035cf" containerName="payload-pre-processing"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108869    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5be301a-1253-4cac-ad1a-03bcd64739e0" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108875    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5be301a-1253-4cac-ad1a-03bcd64739e0" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108882    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108886    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108892    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef1db77e-27c3-4605-84a6-58ff65889c39" containerName="maas-api"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108897    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef1db77e-27c3-4605-84a6-58ff65889c39" containerName="maas-api"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108949    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef1db77e-27c3-4605-84a6-58ff65889c39" containerName="maas-api"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108955    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5be301a-1253-4cac-ad1a-03bcd64739e0" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108964    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="79736d5b-5da9-48b9-b739-a225e04035cf" containerName="payload-pre-processing"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108974    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b58f4381-54b8-4ee0-a185-1bec0ef2c2a5" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108982    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="032179e5-4542-4dcd-bf47-c96596f251d8" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108990    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9bbba8-2263-4760-8106-a38164936770" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.108998    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="faaf8faf-de8a-4f8b-acc2-c6c7ded4c9d5" containerName="main"
Sep 16 13:36:59.109244 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.109006    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="795f6a37-ba3b-4f23-9009-b93c7b4f40eb" containerName="payload-processing"
Sep 16 13:36:59.113252 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.113233    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.115896 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.115874    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-7e7bb4-serving-cert\""
Sep 16 13:36:59.127043 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.127021    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"]
Sep 16 13:36:59.287048 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.287016    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.287177 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.287060    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e506a3-6828-454e-bc09-2555469b3e39-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.287226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.287173    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.287226 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.287208    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mxzm\" (UniqueName: \"kubernetes.io/projected/02e506a3-6828-454e-bc09-2555469b3e39-kube-api-access-9mxzm\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.388195 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.388169    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.388319 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.388203    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mxzm\" (UniqueName: \"kubernetes.io/projected/02e506a3-6828-454e-bc09-2555469b3e39-kube-api-access-9mxzm\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.388319 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.388231    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.388319 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.388291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e506a3-6828-454e-bc09-2555469b3e39-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.389022 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.388999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e506a3-6828-454e-bc09-2555469b3e39-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.390611 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.390579    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.390691 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.390593    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.402501 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.402476    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mxzm\" (UniqueName: \"kubernetes.io/projected/02e506a3-6828-454e-bc09-2555469b3e39-kube-api-access-9mxzm\") pod \"maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.424296 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.424272    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:36:59.716106 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.716080    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"]
Sep 16 13:36:59.717536 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:36:59.717506    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e506a3_6828_454e_bc09_2555469b3e39.slice/crio-509297b8fd2608f271b3cf2db8bf5d396861c00fd3b1a94ad3e18efdb539e271 WatchSource:0}: Error finding container 509297b8fd2608f271b3cf2db8bf5d396861c00fd3b1a94ad3e18efdb539e271: Status 404 returned error can't find the container with id 509297b8fd2608f271b3cf2db8bf5d396861c00fd3b1a94ad3e18efdb539e271
Sep 16 13:36:59.718775 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:36:59.718757    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:37:00.043026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.042948    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"]
Sep 16 13:37:00.047639 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.047623    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.051230 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.051215    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-7e7bb4\""
Sep 16 13:37:00.051696 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.051679    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-7e7bb4-dockercfg-2frn7\""
Sep 16 13:37:00.063771 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.063728    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"]
Sep 16 13:37:00.087571 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.087547    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"]
Sep 16 13:37:00.090686 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.090669    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.103368 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.103346    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"]
Sep 16 13:37:00.194439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.194408    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5603e8-f66a-4b58-a7fb-731faf38f19d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.194553 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.194458    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4xp\" (UniqueName: \"kubernetes.io/projected/5a5603e8-f66a-4b58-a7fb-731faf38f19d-kube-api-access-sg4xp\") pod \"payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.194553 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.194538    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87g9g\" (UniqueName: \"kubernetes.io/projected/39055e3f-e328-4311-b0fc-8293fa1af697-kube-api-access-87g9g\") pod \"payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.194685 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.194571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39055e3f-e328-4311-b0fc-8293fa1af697-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.295082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.295021    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5603e8-f66a-4b58-a7fb-731faf38f19d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.295082 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.295070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4xp\" (UniqueName: \"kubernetes.io/projected/5a5603e8-f66a-4b58-a7fb-731faf38f19d-kube-api-access-sg4xp\") pod \"payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.295225 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.295108    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87g9g\" (UniqueName: \"kubernetes.io/projected/39055e3f-e328-4311-b0fc-8293fa1af697-kube-api-access-87g9g\") pod \"payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.295225 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.295131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39055e3f-e328-4311-b0fc-8293fa1af697-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.295743 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.295725    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5603e8-f66a-4b58-a7fb-731faf38f19d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.295872 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.295856    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39055e3f-e328-4311-b0fc-8293fa1af697-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.306846 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.306825    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4xp\" (UniqueName: \"kubernetes.io/projected/5a5603e8-f66a-4b58-a7fb-731faf38f19d-kube-api-access-sg4xp\") pod \"payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.306946 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.306882    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87g9g\" (UniqueName: \"kubernetes.io/projected/39055e3f-e328-4311-b0fc-8293fa1af697-kube-api-access-87g9g\") pod \"payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.357974 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.357949    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:00.359774 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.359753    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:37:00.401236 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.401191    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:00.633392 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.633366    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"]
Sep 16 13:37:00.635488 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:37:00.635463    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39055e3f_e328_4311_b0fc_8293fa1af697.slice/crio-0ca45ef05ee4c5fe14832b0925fed7c426a92360abd776cc7c187556f86020f6 WatchSource:0}: Error finding container 0ca45ef05ee4c5fe14832b0925fed7c426a92360abd776cc7c187556f86020f6: Status 404 returned error can't find the container with id 0ca45ef05ee4c5fe14832b0925fed7c426a92360abd776cc7c187556f86020f6
Sep 16 13:37:00.671219 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.671193    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv" event={"ID":"02e506a3-6828-454e-bc09-2555469b3e39","Type":"ContainerStarted","Data":"513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab"}
Sep 16 13:37:00.671323 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.671249    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv" event={"ID":"02e506a3-6828-454e-bc09-2555469b3e39","Type":"ContainerStarted","Data":"509297b8fd2608f271b3cf2db8bf5d396861c00fd3b1a94ad3e18efdb539e271"}
Sep 16 13:37:00.671323 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.671281    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:37:00.672523 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.672498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5" event={"ID":"39055e3f-e328-4311-b0fc-8293fa1af697","Type":"ContainerStarted","Data":"0ca45ef05ee4c5fe14832b0925fed7c426a92360abd776cc7c187556f86020f6"}
Sep 16 13:37:00.690122 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.690098    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"]
Sep 16 13:37:00.691362 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:37:00.691333    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5603e8_f66a_4b58_a7fb_731faf38f19d.slice/crio-0f68a55136b3ef05182f7bbe2ecce55833d036ce65c214850a6a7c4ccc572a4b WatchSource:0}: Error finding container 0f68a55136b3ef05182f7bbe2ecce55833d036ce65c214850a6a7c4ccc572a4b: Status 404 returned error can't find the container with id 0f68a55136b3ef05182f7bbe2ecce55833d036ce65c214850a6a7c4ccc572a4b
Sep 16 13:37:00.694855 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:00.694813    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv" podStartSLOduration=1.306263376 podStartE2EDuration="1.694798771s" totalImagesPullingTime="388.535395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:37:00.692396073 +0000 UTC m=+1728.369110510" watchObservedRunningTime="2026-09-16 13:37:00.694798771 +0000 UTC m=+1728.371513205"
Sep 16 13:37:01.679688 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.679642    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5" event={"ID":"39055e3f-e328-4311-b0fc-8293fa1af697","Type":"ContainerStarted","Data":"bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d"}
Sep 16 13:37:01.680128 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.679708    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:01.681438 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.681414    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk" event={"ID":"5a5603e8-f66a-4b58-a7fb-731faf38f19d","Type":"ContainerStarted","Data":"355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95"}
Sep 16 13:37:01.681555 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.681456    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk" event={"ID":"5a5603e8-f66a-4b58-a7fb-731faf38f19d","Type":"ContainerStarted","Data":"0f68a55136b3ef05182f7bbe2ecce55833d036ce65c214850a6a7c4ccc572a4b"}
Sep 16 13:37:01.681606 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.681564    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:01.701848 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.701793    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5" podStartSLOduration=1.7017796299999999 podStartE2EDuration="1.70177963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:37:01.698945379 +0000 UTC m=+1729.375659816" watchObservedRunningTime="2026-09-16 13:37:01.70177963 +0000 UTC m=+1729.378494065"
Sep 16 13:37:01.719922 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:01.719880    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk" podStartSLOduration=1.719866371 podStartE2EDuration="1.719866371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:37:01.718726471 +0000 UTC m=+1729.395440907" watchObservedRunningTime="2026-09-16 13:37:01.719866371 +0000 UTC m=+1729.396580805"
Sep 16 13:37:06.687511 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:06.687478    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:37:12.689209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:12.689175    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:37:12.689666 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:12.689265    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:37:13.951536 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:13.951505    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:37:33.099795 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.099756    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"]
Sep 16 13:37:33.103754 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.103729    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.105771 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.105749    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-8c3cdf-serving-cert\""
Sep 16 13:37:33.114006 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.113981    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"]
Sep 16 13:37:33.144059 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.144040    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6wg2\" (UniqueName: \"kubernetes.io/projected/990ae36f-bb87-4666-9716-d720d4bcc719-kube-api-access-d6wg2\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.144162 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.144077    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990ae36f-bb87-4666-9716-d720d4bcc719-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.144210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.144168    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.144210 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.144205    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.245378 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.245358    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.245498 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.245385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.245498 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.245413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6wg2\" (UniqueName: \"kubernetes.io/projected/990ae36f-bb87-4666-9716-d720d4bcc719-kube-api-access-d6wg2\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.245650 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:37:33.245504    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-serving-cert: secret "maas-api-e2e-shared-b-w3-8c3cdf-serving-cert" not found
Sep 16 13:37:33.245650 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.245515    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990ae36f-bb87-4666-9716-d720d4bcc719-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.245650 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:37:33.245569    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls podName:990ae36f-bb87-4666-9716-d720d4bcc719 nodeName:}" failed. No retries permitted until 2026-09-16 13:37:33.745548035 +0000 UTC m=+1761.422262461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls") pod "maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" (UID: "990ae36f-bb87-4666-9716-d720d4bcc719") : secret "maas-api-e2e-shared-b-w3-8c3cdf-serving-cert" not found
Sep 16 13:37:33.246129 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.246112    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990ae36f-bb87-4666-9716-d720d4bcc719-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.247752 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.247730    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.255346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.255328    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6wg2\" (UniqueName: \"kubernetes.io/projected/990ae36f-bb87-4666-9716-d720d4bcc719-kube-api-access-d6wg2\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.749659 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.749618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:33.752750 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:33.752718    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:34.016445 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:34.016362    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:34.295203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:34.295180    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"]
Sep 16 13:37:34.297899 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:37:34.297870    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990ae36f_bb87_4666_9716_d720d4bcc719.slice/crio-16ae868a6765c0c62f56edfea323c58d944096404fe620d686ce70c3b1f6976c WatchSource:0}: Error finding container 16ae868a6765c0c62f56edfea323c58d944096404fe620d686ce70c3b1f6976c: Status 404 returned error can't find the container with id 16ae868a6765c0c62f56edfea323c58d944096404fe620d686ce70c3b1f6976c
Sep 16 13:37:34.835265 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:34.835234    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" event={"ID":"990ae36f-bb87-4666-9716-d720d4bcc719","Type":"ContainerStarted","Data":"d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b"}
Sep 16 13:37:34.835450 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:34.835285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" event={"ID":"990ae36f-bb87-4666-9716-d720d4bcc719","Type":"ContainerStarted","Data":"16ae868a6765c0c62f56edfea323c58d944096404fe620d686ce70c3b1f6976c"}
Sep 16 13:37:34.835450 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:34.835339    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:37:34.861422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:34.861379    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" podStartSLOduration=1.473852405 podStartE2EDuration="1.861365965s" totalImagesPullingTime="387.51356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:37:34.858050104 +0000 UTC m=+1762.534764540" watchObservedRunningTime="2026-09-16 13:37:34.861365965 +0000 UTC m=+1762.538080400"
Sep 16 13:37:39.954561 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:39.954529    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:37:40.847693 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:37:40.847660    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:38:03.802373 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.802333    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"]
Sep 16 13:38:03.806307 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.806286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:03.808832 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.808810    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-7e7bb4\"/\"openshift-service-ca.crt\""
Sep 16 13:38:03.809503 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.809486    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-7e7bb4\"/\"kube-root-ca.crt\""
Sep 16 13:38:03.809617 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.809501    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7e7bb4\"/\"auth-test-model-7e7bb4-kserve-self-signed-certs\""
Sep 16 13:38:03.809682 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.809586    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7e7bb4\"/\"default-dockercfg-r6wd9\""
Sep 16 13:38:03.816629 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.816584    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"]
Sep 16 13:38:03.907335 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.907302    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-model-cache\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:03.907335 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.907335    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tls-certs\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:03.907627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.907367    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-dshm\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:03.907627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.907415    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tmp-dir\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:03.907627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.907456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt44t\" (UniqueName: \"kubernetes.io/projected/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-kube-api-access-mt44t\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:03.907627 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:03.907506    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-home\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.007983 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.007952    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt44t\" (UniqueName: \"kubernetes.io/projected/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-kube-api-access-mt44t\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008010    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-home\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008026    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-model-cache\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tls-certs\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-dshm\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008136 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008090    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tmp-dir\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008481    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-model-cache\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008505 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008496    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-home\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.008629 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.008561    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tmp-dir\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.010423 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.010405    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-dshm\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.010504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.010488    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tls-certs\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.016027 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.016007    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt44t\" (UniqueName: \"kubernetes.io/projected/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-kube-api-access-mt44t\") pod \"auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") " pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.115581 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.115514    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:04.394203 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.394179    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"]
Sep 16 13:38:04.396518 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:38:04.396489    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cbbdd0e_78e4_44ab_92f2_aa1e3d32c39a.slice/crio-d710e3e70173d8a50cdfa109fb8f2d67f12e9e930d8d6967300d42da903f8022 WatchSource:0}: Error finding container d710e3e70173d8a50cdfa109fb8f2d67f12e9e930d8d6967300d42da903f8022: Status 404 returned error can't find the container with id d710e3e70173d8a50cdfa109fb8f2d67f12e9e930d8d6967300d42da903f8022
Sep 16 13:38:04.984926 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.984893    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h" event={"ID":"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a","Type":"ContainerStarted","Data":"e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84"}
Sep 16 13:38:04.984926 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:04.984931    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h" event={"ID":"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a","Type":"ContainerStarted","Data":"d710e3e70173d8a50cdfa109fb8f2d67f12e9e930d8d6967300d42da903f8022"}
Sep 16 13:38:05.008328 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:05.008269    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h" podStartSLOduration=2.008253867 podStartE2EDuration="2.008253867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:38:05.006590871 +0000 UTC m=+1792.683305305" watchObservedRunningTime="2026-09-16 13:38:05.008253867 +0000 UTC m=+1792.684968302"
Sep 16 13:38:08.752589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:08.752544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:38:12.893581 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:12.893550    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:38:12.898411 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:12.898386    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:38:14.116241 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:14.116209    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:14.116676 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:14.116255    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:14.129046 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:14.129017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:15.043002 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:15.042972    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:38:31.360254 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.360217    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"]
Sep 16 13:38:31.363655 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.363632    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.378391 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.378365    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8c3cdf\"/\"auth-test-model-8c3cdf-kserve-self-signed-certs\""
Sep 16 13:38:31.378507 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.378399    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8c3cdf\"/\"openshift-service-ca.crt\""
Sep 16 13:38:31.378574 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.378504    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8c3cdf\"/\"kube-root-ca.crt\""
Sep 16 13:38:31.378574 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.378564    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8c3cdf\"/\"default-dockercfg-xhlkc\""
Sep 16 13:38:31.384112 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.384090    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"]
Sep 16 13:38:31.438974 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.438948    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-dshm\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.439073 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.438982    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-home\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.439073 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.439013    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tmp-dir\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.439146 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.439074    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tls-certs\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.439146 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.439127    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-model-cache\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.439217 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.439148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9x8p\" (UniqueName: \"kubernetes.io/projected/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-kube-api-access-h9x8p\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540036 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540006    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9x8p\" (UniqueName: \"kubernetes.io/projected/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-kube-api-access-h9x8p\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540197 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-dshm\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540197 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540108    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-home\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540197 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540141    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tmp-dir\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540197 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540167    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tls-certs\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540216    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-model-cache\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540545 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540525    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-home\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540641 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tmp-dir\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.540705 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.540636    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-model-cache\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.542457 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.542433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-dshm\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.542733 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.542719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tls-certs\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.553807 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.553787    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9x8p\" (UniqueName: \"kubernetes.io/projected/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-kube-api-access-h9x8p\") pod \"auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") " pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.672589 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.672556    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:31.968385 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:31.968360    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"]
Sep 16 13:38:31.970634 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:38:31.970585    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff02c1e2_cbd9_4719_8026_ff2f4d33ea3f.slice/crio-8b5ced14a6b2cde2d66486502b68e7c17db05bd2c061b9a41bffded2fab27788 WatchSource:0}: Error finding container 8b5ced14a6b2cde2d66486502b68e7c17db05bd2c061b9a41bffded2fab27788: Status 404 returned error can't find the container with id 8b5ced14a6b2cde2d66486502b68e7c17db05bd2c061b9a41bffded2fab27788
Sep 16 13:38:32.111671 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:32.111637    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4" event={"ID":"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f","Type":"ContainerStarted","Data":"0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273"}
Sep 16 13:38:32.111821 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:32.111680    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4" event={"ID":"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f","Type":"ContainerStarted","Data":"8b5ced14a6b2cde2d66486502b68e7c17db05bd2c061b9a41bffded2fab27788"}
Sep 16 13:38:32.150621 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:32.150558    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4" podStartSLOduration=1.150544357 podStartE2EDuration="1.150544357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:38:32.149110825 +0000 UTC m=+1819.825825255" watchObservedRunningTime="2026-09-16 13:38:32.150544357 +0000 UTC m=+1819.827258792"
Sep 16 13:38:37.464394 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:37.464357    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:38:41.672790 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:41.672758    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:41.673169 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:41.672835    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:41.686255 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:41.686225    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:38:42.177570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:38:42.177543    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:39:46.854042 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:39:46.854010    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:41:25.246424 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:41:25.246343    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:43:12.921938 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:43:12.921909    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:43:12.928163 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:43:12.928128    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:43:38.691469 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:43:38.691432    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:44:07.301124 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.301050    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"]
Sep 16 13:44:07.304779 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.304762    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.308325 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.308306    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-804072-serving-cert\""
Sep 16 13:44:07.329473 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.329451    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"]
Sep 16 13:44:07.408673 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.408652    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-maas-api-tls\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.408795 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.408715    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49hhb\" (UniqueName: \"kubernetes.io/projected/b65873d1-c729-4652-8d5c-44f202142776-kube-api-access-49hhb\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.408795 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.408786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-metrics-certs\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.408891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.408806    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65873d1-c729-4652-8d5c-44f202142776-openshift-service-ca\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.509922 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.509889    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-maas-api-tls\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.510067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.509986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49hhb\" (UniqueName: \"kubernetes.io/projected/b65873d1-c729-4652-8d5c-44f202142776-kube-api-access-49hhb\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.510067 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.510052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-metrics-certs\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.510199 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.510084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65873d1-c729-4652-8d5c-44f202142776-openshift-service-ca\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.510726 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.510704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65873d1-c729-4652-8d5c-44f202142776-openshift-service-ca\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.512311 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.512292    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-maas-api-tls\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.512409 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.512356    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-metrics-certs\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.524523 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.524500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49hhb\" (UniqueName: \"kubernetes.io/projected/b65873d1-c729-4652-8d5c-44f202142776-kube-api-access-49hhb\") pod \"maas-api-e2e-api-b-804072-867594db7-f7q9t\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.614807 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.614749    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:07.894741 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.894714    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"]
Sep 16 13:44:07.897972 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:44:07.897943    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65873d1_c729_4652_8d5c_44f202142776.slice/crio-57b1293966a6375b225a8c0b2aa0aab81a484f0647ee57f1224ac19da0a855e7 WatchSource:0}: Error finding container 57b1293966a6375b225a8c0b2aa0aab81a484f0647ee57f1224ac19da0a855e7: Status 404 returned error can't find the container with id 57b1293966a6375b225a8c0b2aa0aab81a484f0647ee57f1224ac19da0a855e7
Sep 16 13:44:07.899239 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:07.899220    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:44:08.754443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:08.754404    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t" event={"ID":"b65873d1-c729-4652-8d5c-44f202142776","Type":"ContainerStarted","Data":"2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6"}
Sep 16 13:44:08.754832 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:08.754486    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t" event={"ID":"b65873d1-c729-4652-8d5c-44f202142776","Type":"ContainerStarted","Data":"57b1293966a6375b225a8c0b2aa0aab81a484f0647ee57f1224ac19da0a855e7"}
Sep 16 13:44:08.754832 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:08.754616    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:08.798703 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:08.798649    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t" podStartSLOduration=1.448856137 podStartE2EDuration="1.798630327s" totalImagesPullingTime="349.77419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:44:08.796477874 +0000 UTC m=+2156.473192309" watchObservedRunningTime="2026-09-16 13:44:08.798630327 +0000 UTC m=+2156.475344780"
Sep 16 13:44:14.764817 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:14.764785    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:44:15.032798 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:15.032709    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:44:47.062765 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:44:47.062728    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:45:00.212655 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.212618    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"]
Sep 16 13:45:00.216291 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.216267    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:00.217844 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.217815    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"]
Sep 16 13:45:00.219088 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.219056    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nvp5m\""
Sep 16 13:45:00.221635 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.221589    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"]
Sep 16 13:45:00.221729 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.221661    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:00.225401 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.225384    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:00.247858 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.247832    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"]
Sep 16 13:45:00.253063 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.253037    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"]
Sep 16 13:45:00.254925 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.254905    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"]
Sep 16 13:45:00.271732 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.271701    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blcp2\" (UniqueName: \"kubernetes.io/projected/5944edf6-d977-4a86-b827-fe8f2a01382d-kube-api-access-blcp2\") pod \"maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp\" (UID: \"5944edf6-d977-4a86-b827-fe8f2a01382d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:00.271816 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.271752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxkgt\" (UniqueName: \"kubernetes.io/projected/70051f72-b766-4b03-ae3c-a36b055cb25b-kube-api-access-dxkgt\") pod \"maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v\" (UID: \"70051f72-b766-4b03-ae3c-a36b055cb25b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:00.271816 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.271792    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpcc6\" (UniqueName: \"kubernetes.io/projected/f90b708c-5543-4462-b90e-110deb34cb97-kube-api-access-dpcc6\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw\" (UID: \"f90b708c-5543-4462-b90e-110deb34cb97\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:00.372644 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.372619    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blcp2\" (UniqueName: \"kubernetes.io/projected/5944edf6-d977-4a86-b827-fe8f2a01382d-kube-api-access-blcp2\") pod \"maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp\" (UID: \"5944edf6-d977-4a86-b827-fe8f2a01382d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:00.372737 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.372650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxkgt\" (UniqueName: \"kubernetes.io/projected/70051f72-b766-4b03-ae3c-a36b055cb25b-kube-api-access-dxkgt\") pod \"maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v\" (UID: \"70051f72-b766-4b03-ae3c-a36b055cb25b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:00.372737 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.372673    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpcc6\" (UniqueName: \"kubernetes.io/projected/f90b708c-5543-4462-b90e-110deb34cb97-kube-api-access-dpcc6\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw\" (UID: \"f90b708c-5543-4462-b90e-110deb34cb97\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:00.381185 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.381153    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blcp2\" (UniqueName: \"kubernetes.io/projected/5944edf6-d977-4a86-b827-fe8f2a01382d-kube-api-access-blcp2\") pod \"maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp\" (UID: \"5944edf6-d977-4a86-b827-fe8f2a01382d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:00.381432 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.381411    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpcc6\" (UniqueName: \"kubernetes.io/projected/f90b708c-5543-4462-b90e-110deb34cb97-kube-api-access-dpcc6\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw\" (UID: \"f90b708c-5543-4462-b90e-110deb34cb97\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:00.381499 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.381438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxkgt\" (UniqueName: \"kubernetes.io/projected/70051f72-b766-4b03-ae3c-a36b055cb25b-kube-api-access-dxkgt\") pod \"maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v\" (UID: \"70051f72-b766-4b03-ae3c-a36b055cb25b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:00.528801 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.528741    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:00.535438 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.535418    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:00.541826 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.541804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:00.815202 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.815173    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"]
Sep 16 13:45:00.817119 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:45:00.817032    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90b708c_5543_4462_b90e_110deb34cb97.slice/crio-0f27a464aa2fac54570837639c39463e0a8c6a73f1ee9febd75d0194f5bfc9a6 WatchSource:0}: Error finding container 0f27a464aa2fac54570837639c39463e0a8c6a73f1ee9febd75d0194f5bfc9a6: Status 404 returned error can't find the container with id 0f27a464aa2fac54570837639c39463e0a8c6a73f1ee9febd75d0194f5bfc9a6
Sep 16 13:45:00.818666 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.818473    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"]
Sep 16 13:45:00.820633 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:45:00.820609    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70051f72_b766_4b03_ae3c_a36b055cb25b.slice/crio-33fbc1264fbe74a0077f005b466fefb0b2feb9d074243931499dc229402a4c15 WatchSource:0}: Error finding container 33fbc1264fbe74a0077f005b466fefb0b2feb9d074243931499dc229402a4c15: Status 404 returned error can't find the container with id 33fbc1264fbe74a0077f005b466fefb0b2feb9d074243931499dc229402a4c15
Sep 16 13:45:00.834379 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:00.834352    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"]
Sep 16 13:45:00.834570 ip-10-0-143-82 kubenswrapper[2556]: W0916 13:45:00.834544    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5944edf6_d977_4a86_b827_fe8f2a01382d.slice/crio-f1c94475ae132df196c53362b3035e5fd9837865168e0768553a5290d126d7c9 WatchSource:0}: Error finding container f1c94475ae132df196c53362b3035e5fd9837865168e0768553a5290d126d7c9: Status 404 returned error can't find the container with id f1c94475ae132df196c53362b3035e5fd9837865168e0768553a5290d126d7c9
Sep 16 13:45:01.019697 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:01.019660    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp" event={"ID":"5944edf6-d977-4a86-b827-fe8f2a01382d","Type":"ContainerStarted","Data":"f1c94475ae132df196c53362b3035e5fd9837865168e0768553a5290d126d7c9"}
Sep 16 13:45:01.020834 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:01.020811    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v" event={"ID":"70051f72-b766-4b03-ae3c-a36b055cb25b","Type":"ContainerStarted","Data":"33fbc1264fbe74a0077f005b466fefb0b2feb9d074243931499dc229402a4c15"}
Sep 16 13:45:01.021908 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:01.021891    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw" event={"ID":"f90b708c-5543-4462-b90e-110deb34cb97","Type":"ContainerStarted","Data":"0f27a464aa2fac54570837639c39463e0a8c6a73f1ee9febd75d0194f5bfc9a6"}
Sep 16 13:45:02.028919 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:02.028880    2556 generic.go:356] "Generic (PLEG): container finished" podID="70051f72-b766-4b03-ae3c-a36b055cb25b" containerID="25c87e90eeeec7daf5c74698c70dd56d6ff0bd24a827fa5564fe54259a2bfc6c" exitCode=0
Sep 16 13:45:02.029229 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:02.028942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v" event={"ID":"70051f72-b766-4b03-ae3c-a36b055cb25b","Type":"ContainerDied","Data":"25c87e90eeeec7daf5c74698c70dd56d6ff0bd24a827fa5564fe54259a2bfc6c"}
Sep 16 13:45:02.030580 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:02.030560    2556 generic.go:356] "Generic (PLEG): container finished" podID="f90b708c-5543-4462-b90e-110deb34cb97" containerID="fc6c1e7404058b5a21f4f1709dcbe2bdc38eba96332a5168110a68feb651d1e8" exitCode=0
Sep 16 13:45:02.030687 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:02.030587    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw" event={"ID":"f90b708c-5543-4462-b90e-110deb34cb97","Type":"ContainerDied","Data":"fc6c1e7404058b5a21f4f1709dcbe2bdc38eba96332a5168110a68feb651d1e8"}
Sep 16 13:45:02.032103 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:02.032079    2556 generic.go:356] "Generic (PLEG): container finished" podID="5944edf6-d977-4a86-b827-fe8f2a01382d" containerID="34e0a48e480152e774b81517708525a6a22c99cb8974b29ff98644808b3e9785" exitCode=0
Sep 16 13:45:02.032202 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:02.032155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp" event={"ID":"5944edf6-d977-4a86-b827-fe8f2a01382d","Type":"ContainerDied","Data":"34e0a48e480152e774b81517708525a6a22c99cb8974b29ff98644808b3e9785"}
Sep 16 13:45:03.184301 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.184262    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:03.233494 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.233468    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:03.236888 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.236866    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:03.298114 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.298023    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxkgt\" (UniqueName: \"kubernetes.io/projected/70051f72-b766-4b03-ae3c-a36b055cb25b-kube-api-access-dxkgt\") pod \"70051f72-b766-4b03-ae3c-a36b055cb25b\" (UID: \"70051f72-b766-4b03-ae3c-a36b055cb25b\") "
Sep 16 13:45:03.298280 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.298186    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpcc6\" (UniqueName: \"kubernetes.io/projected/f90b708c-5543-4462-b90e-110deb34cb97-kube-api-access-dpcc6\") pod \"f90b708c-5543-4462-b90e-110deb34cb97\" (UID: \"f90b708c-5543-4462-b90e-110deb34cb97\") "
Sep 16 13:45:03.298280 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.298235    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blcp2\" (UniqueName: \"kubernetes.io/projected/5944edf6-d977-4a86-b827-fe8f2a01382d-kube-api-access-blcp2\") pod \"5944edf6-d977-4a86-b827-fe8f2a01382d\" (UID: \"5944edf6-d977-4a86-b827-fe8f2a01382d\") "
Sep 16 13:45:03.301800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.301663    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90b708c-5543-4462-b90e-110deb34cb97-kube-api-access-dpcc6" (OuterVolumeSpecName: "kube-api-access-dpcc6") pod "f90b708c-5543-4462-b90e-110deb34cb97" (UID: "f90b708c-5543-4462-b90e-110deb34cb97"). InnerVolumeSpecName "kube-api-access-dpcc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:03.301800 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.301744    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70051f72-b766-4b03-ae3c-a36b055cb25b-kube-api-access-dxkgt" (OuterVolumeSpecName: "kube-api-access-dxkgt") pod "70051f72-b766-4b03-ae3c-a36b055cb25b" (UID: "70051f72-b766-4b03-ae3c-a36b055cb25b"). InnerVolumeSpecName "kube-api-access-dxkgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:03.302010 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.301849    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5944edf6-d977-4a86-b827-fe8f2a01382d-kube-api-access-blcp2" (OuterVolumeSpecName: "kube-api-access-blcp2") pod "5944edf6-d977-4a86-b827-fe8f2a01382d" (UID: "5944edf6-d977-4a86-b827-fe8f2a01382d"). InnerVolumeSpecName "kube-api-access-blcp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:03.399231 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.399202    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpcc6\" (UniqueName: \"kubernetes.io/projected/f90b708c-5543-4462-b90e-110deb34cb97-kube-api-access-dpcc6\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:03.399231 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.399229    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blcp2\" (UniqueName: \"kubernetes.io/projected/5944edf6-d977-4a86-b827-fe8f2a01382d-kube-api-access-blcp2\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:03.399389 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:03.399245    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxkgt\" (UniqueName: \"kubernetes.io/projected/70051f72-b766-4b03-ae3c-a36b055cb25b-kube-api-access-dxkgt\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:04.023631 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.023576    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"]
Sep 16 13:45:04.023822 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.023787    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t" podUID="b65873d1-c729-4652-8d5c-44f202142776" containerName="maas-api" containerID="cri-o://2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6" gracePeriod=30
Sep 16 13:45:04.044163 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.044132    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"
Sep 16 13:45:04.044289 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.044139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v" event={"ID":"70051f72-b766-4b03-ae3c-a36b055cb25b","Type":"ContainerDied","Data":"33fbc1264fbe74a0077f005b466fefb0b2feb9d074243931499dc229402a4c15"}
Sep 16 13:45:04.044355 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.044289    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33fbc1264fbe74a0077f005b466fefb0b2feb9d074243931499dc229402a4c15"
Sep 16 13:45:04.045877 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.045763    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"
Sep 16 13:45:04.045877 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.045809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw" event={"ID":"f90b708c-5543-4462-b90e-110deb34cb97","Type":"ContainerDied","Data":"0f27a464aa2fac54570837639c39463e0a8c6a73f1ee9febd75d0194f5bfc9a6"}
Sep 16 13:45:04.046066 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.045885    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f27a464aa2fac54570837639c39463e0a8c6a73f1ee9febd75d0194f5bfc9a6"
Sep 16 13:45:04.048016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.047994    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp" event={"ID":"5944edf6-d977-4a86-b827-fe8f2a01382d","Type":"ContainerDied","Data":"f1c94475ae132df196c53362b3035e5fd9837865168e0768553a5290d126d7c9"}
Sep 16 13:45:04.048129 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.048038    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1c94475ae132df196c53362b3035e5fd9837865168e0768553a5290d126d7c9"
Sep 16 13:45:04.048129 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.048096    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp"
Sep 16 13:45:04.090711 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.090679    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"]
Sep 16 13:45:04.095810 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.095781    2556 status_manager.go:1073] "Failed to delete status for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v" err="pods \"maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v\" not found"
Sep 16 13:45:04.098146 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.098122    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-804072-29826105-6sd7v"]
Sep 16 13:45:04.277261 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.277212    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:45:04.408016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.407981    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-metrics-certs\") pod \"b65873d1-c729-4652-8d5c-44f202142776\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") "
Sep 16 13:45:04.408213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.408029    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65873d1-c729-4652-8d5c-44f202142776-openshift-service-ca\") pod \"b65873d1-c729-4652-8d5c-44f202142776\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") "
Sep 16 13:45:04.408213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.408066    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-maas-api-tls\") pod \"b65873d1-c729-4652-8d5c-44f202142776\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") "
Sep 16 13:45:04.408213 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.408101    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49hhb\" (UniqueName: \"kubernetes.io/projected/b65873d1-c729-4652-8d5c-44f202142776-kube-api-access-49hhb\") pod \"b65873d1-c729-4652-8d5c-44f202142776\" (UID: \"b65873d1-c729-4652-8d5c-44f202142776\") "
Sep 16 13:45:04.408504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.408469    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b65873d1-c729-4652-8d5c-44f202142776-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b65873d1-c729-4652-8d5c-44f202142776" (UID: "b65873d1-c729-4652-8d5c-44f202142776"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:45:04.410587 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.410556    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b65873d1-c729-4652-8d5c-44f202142776" (UID: "b65873d1-c729-4652-8d5c-44f202142776"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:45:04.410723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.410641    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b65873d1-c729-4652-8d5c-44f202142776" (UID: "b65873d1-c729-4652-8d5c-44f202142776"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:45:04.410723 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.410664    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b65873d1-c729-4652-8d5c-44f202142776-kube-api-access-49hhb" (OuterVolumeSpecName: "kube-api-access-49hhb") pod "b65873d1-c729-4652-8d5c-44f202142776" (UID: "b65873d1-c729-4652-8d5c-44f202142776"). InnerVolumeSpecName "kube-api-access-49hhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:04.509024 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.509000    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49hhb\" (UniqueName: \"kubernetes.io/projected/b65873d1-c729-4652-8d5c-44f202142776-kube-api-access-49hhb\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:04.509024 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.509023    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:04.509150 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.509036    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65873d1-c729-4652-8d5c-44f202142776-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:04.509150 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.509045    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65873d1-c729-4652-8d5c-44f202142776-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:04.800423 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:04.800394    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70051f72-b766-4b03-ae3c-a36b055cb25b" path="/var/lib/kubelet/pods/70051f72-b766-4b03-ae3c-a36b055cb25b/volumes"
Sep 16 13:45:05.054835 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.054748    2556 generic.go:356] "Generic (PLEG): container finished" podID="b65873d1-c729-4652-8d5c-44f202142776" containerID="2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6" exitCode=0
Sep 16 13:45:05.054835 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.054789    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"
Sep 16 13:45:05.054835 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.054798    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t" event={"ID":"b65873d1-c729-4652-8d5c-44f202142776","Type":"ContainerDied","Data":"2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6"}
Sep 16 13:45:05.055084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.054841    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t" event={"ID":"b65873d1-c729-4652-8d5c-44f202142776","Type":"ContainerDied","Data":"57b1293966a6375b225a8c0b2aa0aab81a484f0647ee57f1224ac19da0a855e7"}
Sep 16 13:45:05.055084 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.054862    2556 scope.go:117] "RemoveContainer" containerID="2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6"
Sep 16 13:45:05.063874 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.063855    2556 scope.go:117] "RemoveContainer" containerID="2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6"
Sep 16 13:45:05.064154 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:45:05.064134    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6\": container with ID starting with 2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6 not found: ID does not exist" containerID="2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6"
Sep 16 13:45:05.064199 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.064162    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6"} err="failed to get container status \"2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6\": rpc error: code = NotFound desc = could not find container \"2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6\": container with ID starting with 2ce02ccc2b1c6f7da093d2234ddddd89e276b13458fbd5bde165cf1f76ceffa6 not found: ID does not exist"
Sep 16 13:45:05.073910 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.073882    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"]
Sep 16 13:45:05.076741 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:05.076720    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-804072-867594db7-f7q9t"]
Sep 16 13:45:06.801492 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:06.801459    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b65873d1-c729-4652-8d5c-44f202142776" path="/var/lib/kubelet/pods/b65873d1-c729-4652-8d5c-44f202142776/volumes"
Sep 16 13:45:10.045177 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:10.045146    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:45:22.756756 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:22.756721    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"]
Sep 16 13:45:22.762958 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:22.762934    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b0514e-29826105-9ntcw"]
Sep 16 13:45:22.801849 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:22.801819    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f90b708c-5543-4462-b90e-110deb34cb97" path="/var/lib/kubelet/pods/f90b708c-5543-4462-b90e-110deb34cb97/volumes"
Sep 16 13:45:22.820941 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:22.820908    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"]
Sep 16 13:45:22.821151 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:22.821125    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7" podUID="01d67d28-55ed-443c-9aab-43476531e2d7" containerName="payload-pre-processing" containerID="cri-o://98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173" gracePeriod=30
Sep 16 13:45:23.083454 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.083434    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:45:23.142496 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.142465    2556 generic.go:356] "Generic (PLEG): container finished" podID="01d67d28-55ed-443c-9aab-43476531e2d7" containerID="98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173" exitCode=0
Sep 16 13:45:23.142643 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.142510    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"
Sep 16 13:45:23.142643 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.142549    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7" event={"ID":"01d67d28-55ed-443c-9aab-43476531e2d7","Type":"ContainerDied","Data":"98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173"}
Sep 16 13:45:23.142643 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.142629    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7" event={"ID":"01d67d28-55ed-443c-9aab-43476531e2d7","Type":"ContainerDied","Data":"75b497d322482e700f5e32fc252c1f74f0d59461ca10f5cd7f142b2d7c96e524"}
Sep 16 13:45:23.142765 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.142662    2556 scope.go:117] "RemoveContainer" containerID="98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173"
Sep 16 13:45:23.151016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.150999    2556 scope.go:117] "RemoveContainer" containerID="98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173"
Sep 16 13:45:23.151283 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:45:23.151260    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173\": container with ID starting with 98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173 not found: ID does not exist" containerID="98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173"
Sep 16 13:45:23.151346 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.151292    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173"} err="failed to get container status \"98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173\": rpc error: code = NotFound desc = could not find container \"98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173\": container with ID starting with 98f2886d0dd23383a87295bf93f2b31c485a2d565adbf7db6e827360da8d6173 not found: ID does not exist"
Sep 16 13:45:23.170209 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.170189    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d67d28-55ed-443c-9aab-43476531e2d7-plugins-config-volume\") pod \"01d67d28-55ed-443c-9aab-43476531e2d7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") "
Sep 16 13:45:23.170273 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.170255    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfgqw\" (UniqueName: \"kubernetes.io/projected/01d67d28-55ed-443c-9aab-43476531e2d7-kube-api-access-sfgqw\") pod \"01d67d28-55ed-443c-9aab-43476531e2d7\" (UID: \"01d67d28-55ed-443c-9aab-43476531e2d7\") "
Sep 16 13:45:23.170522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.170502    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01d67d28-55ed-443c-9aab-43476531e2d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "01d67d28-55ed-443c-9aab-43476531e2d7" (UID: "01d67d28-55ed-443c-9aab-43476531e2d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:45:23.172251 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.172233    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d67d28-55ed-443c-9aab-43476531e2d7-kube-api-access-sfgqw" (OuterVolumeSpecName: "kube-api-access-sfgqw") pod "01d67d28-55ed-443c-9aab-43476531e2d7" (UID: "01d67d28-55ed-443c-9aab-43476531e2d7"). InnerVolumeSpecName "kube-api-access-sfgqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:23.271369 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.271346    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d67d28-55ed-443c-9aab-43476531e2d7-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:23.271369 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.271366    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfgqw\" (UniqueName: \"kubernetes.io/projected/01d67d28-55ed-443c-9aab-43476531e2d7-kube-api-access-sfgqw\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:23.465047 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.465017    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"]
Sep 16 13:45:23.470866 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:23.470844    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b0514e-85f6f4765d-j5mf7"]
Sep 16 13:45:24.800124 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:24.800089    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01d67d28-55ed-443c-9aab-43476531e2d7" path="/var/lib/kubelet/pods/01d67d28-55ed-443c-9aab-43476531e2d7/volumes"
Sep 16 13:45:27.664026 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:27.663992    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:45:41.307176 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.307058    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"]
Sep 16 13:45:41.307613 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.307286    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl" podUID="66fc331d-fd62-474d-8427-215860f0a583" containerName="payload-pre-processing" containerID="cri-o://dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6" gracePeriod=30
Sep 16 13:45:41.567894 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.567836    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:45:41.717803 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.717766    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66fc331d-fd62-474d-8427-215860f0a583-plugins-config-volume\") pod \"66fc331d-fd62-474d-8427-215860f0a583\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") "
Sep 16 13:45:41.717981 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.717820    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5xwq\" (UniqueName: \"kubernetes.io/projected/66fc331d-fd62-474d-8427-215860f0a583-kube-api-access-c5xwq\") pod \"66fc331d-fd62-474d-8427-215860f0a583\" (UID: \"66fc331d-fd62-474d-8427-215860f0a583\") "
Sep 16 13:45:41.718219 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.718193    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66fc331d-fd62-474d-8427-215860f0a583-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66fc331d-fd62-474d-8427-215860f0a583" (UID: "66fc331d-fd62-474d-8427-215860f0a583"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:45:41.719865 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.719847    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66fc331d-fd62-474d-8427-215860f0a583-kube-api-access-c5xwq" (OuterVolumeSpecName: "kube-api-access-c5xwq") pod "66fc331d-fd62-474d-8427-215860f0a583" (UID: "66fc331d-fd62-474d-8427-215860f0a583"). InnerVolumeSpecName "kube-api-access-c5xwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:41.818450 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.818392    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66fc331d-fd62-474d-8427-215860f0a583-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:41.818450 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:41.818413    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5xwq\" (UniqueName: \"kubernetes.io/projected/66fc331d-fd62-474d-8427-215860f0a583-kube-api-access-c5xwq\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:45:42.244891 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.244855    2556 generic.go:356] "Generic (PLEG): container finished" podID="66fc331d-fd62-474d-8427-215860f0a583" containerID="dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6" exitCode=0
Sep 16 13:45:42.245037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.244894    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"
Sep 16 13:45:42.245037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.244933    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl" event={"ID":"66fc331d-fd62-474d-8427-215860f0a583","Type":"ContainerDied","Data":"dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6"}
Sep 16 13:45:42.245037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.244987    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl" event={"ID":"66fc331d-fd62-474d-8427-215860f0a583","Type":"ContainerDied","Data":"1f7a070655641fe4b8e96620b917f52190001a499b4461afc91aeca5b958fd51"}
Sep 16 13:45:42.245037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.245021    2556 scope.go:117] "RemoveContainer" containerID="dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6"
Sep 16 13:45:42.254407 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.254391    2556 scope.go:117] "RemoveContainer" containerID="dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6"
Sep 16 13:45:42.254689 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:45:42.254671    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6\": container with ID starting with dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6 not found: ID does not exist" containerID="dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6"
Sep 16 13:45:42.254787 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.254698    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6"} err="failed to get container status \"dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6\": rpc error: code = NotFound desc = could not find container \"dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6\": container with ID starting with dff904b1a18de7ad329bdd5327c7005fba87a338b14f331b814484c132ad93e6 not found: ID does not exist"
Sep 16 13:45:42.266695 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.266675    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"]
Sep 16 13:45:42.273686 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.273664    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7841d9-65c5fdcf5d-w5stl"]
Sep 16 13:45:42.799962 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:42.799930    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66fc331d-fd62-474d-8427-215860f0a583" path="/var/lib/kubelet/pods/66fc331d-fd62-474d-8427-215860f0a583/volumes"
Sep 16 13:45:45.344028 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:45:45.343995    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:48:12.953917 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:48:12.953882    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:48:12.961339 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:48:12.961316    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:51:13.126183 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:51:13.126148    2556 scope.go:117] "RemoveContainer" containerID="fc6c1e7404058b5a21f4f1709dcbe2bdc38eba96332a5168110a68feb651d1e8"
Sep 16 13:51:13.134518 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:51:13.134494    2556 scope.go:117] "RemoveContainer" containerID="25c87e90eeeec7daf5c74698c70dd56d6ff0bd24a827fa5564fe54259a2bfc6c"
Sep 16 13:53:12.984441 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:53:12.984350    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:53:12.997482 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:53:12.997459    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:54:17.332171 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:17.332134    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"]
Sep 16 13:54:18.305087 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:18.305057    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"]
Sep 16 13:54:21.250154 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:21.250118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:54:28.904748 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:28.904714    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"]
Sep 16 13:54:28.905220 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:28.904955    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv" podUID="02e506a3-6828-454e-bc09-2555469b3e39" containerName="maas-api" containerID="cri-o://513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab" gracePeriod=30
Sep 16 13:54:28.966440 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:28.966409    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"]
Sep 16 13:54:28.966671 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:28.966641    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5" podUID="39055e3f-e328-4311-b0fc-8293fa1af697" containerName="payload-processing" containerID="cri-o://bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d" gracePeriod=30
Sep 16 13:54:28.973099 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:28.973033    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"]
Sep 16 13:54:28.973248 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:28.973221    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk" podUID="5a5603e8-f66a-4b58-a7fb-731faf38f19d" containerName="payload-pre-processing" containerID="cri-o://355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95" gracePeriod=30
Sep 16 13:54:29.238439 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.238412    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:54:29.251785 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.251762    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:54:29.261745 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.261724    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:54:29.318320 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318295    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39055e3f-e328-4311-b0fc-8293fa1af697-plugins-config-volume\") pod \"39055e3f-e328-4311-b0fc-8293fa1af697\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") "
Sep 16 13:54:29.318443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318377    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-metrics-certs\") pod \"02e506a3-6828-454e-bc09-2555469b3e39\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") "
Sep 16 13:54:29.318443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318396    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-maas-api-tls\") pod \"02e506a3-6828-454e-bc09-2555469b3e39\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") "
Sep 16 13:54:29.318443 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318411    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e506a3-6828-454e-bc09-2555469b3e39-openshift-service-ca\") pod \"02e506a3-6828-454e-bc09-2555469b3e39\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") "
Sep 16 13:54:29.318611 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318469    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mxzm\" (UniqueName: \"kubernetes.io/projected/02e506a3-6828-454e-bc09-2555469b3e39-kube-api-access-9mxzm\") pod \"02e506a3-6828-454e-bc09-2555469b3e39\" (UID: \"02e506a3-6828-454e-bc09-2555469b3e39\") "
Sep 16 13:54:29.318674 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318633    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87g9g\" (UniqueName: \"kubernetes.io/projected/39055e3f-e328-4311-b0fc-8293fa1af697-kube-api-access-87g9g\") pod \"39055e3f-e328-4311-b0fc-8293fa1af697\" (UID: \"39055e3f-e328-4311-b0fc-8293fa1af697\") "
Sep 16 13:54:29.318836 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318800    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39055e3f-e328-4311-b0fc-8293fa1af697-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39055e3f-e328-4311-b0fc-8293fa1af697" (UID: "39055e3f-e328-4311-b0fc-8293fa1af697"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:54:29.318916 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.318824    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02e506a3-6828-454e-bc09-2555469b3e39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "02e506a3-6828-454e-bc09-2555469b3e39" (UID: "02e506a3-6828-454e-bc09-2555469b3e39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:54:29.319061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.319035    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39055e3f-e328-4311-b0fc-8293fa1af697-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.319061 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.319063    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e506a3-6828-454e-bc09-2555469b3e39-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.320669 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.320641    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "02e506a3-6828-454e-bc09-2555469b3e39" (UID: "02e506a3-6828-454e-bc09-2555469b3e39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:54:29.320772 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.320715    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39055e3f-e328-4311-b0fc-8293fa1af697-kube-api-access-87g9g" (OuterVolumeSpecName: "kube-api-access-87g9g") pod "39055e3f-e328-4311-b0fc-8293fa1af697" (UID: "39055e3f-e328-4311-b0fc-8293fa1af697"). InnerVolumeSpecName "kube-api-access-87g9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:54:29.320772 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.320715    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "02e506a3-6828-454e-bc09-2555469b3e39" (UID: "02e506a3-6828-454e-bc09-2555469b3e39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:54:29.321037 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.321016    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e506a3-6828-454e-bc09-2555469b3e39-kube-api-access-9mxzm" (OuterVolumeSpecName: "kube-api-access-9mxzm") pod "02e506a3-6828-454e-bc09-2555469b3e39" (UID: "02e506a3-6828-454e-bc09-2555469b3e39"). InnerVolumeSpecName "kube-api-access-9mxzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:54:29.419818 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.419801    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5603e8-f66a-4b58-a7fb-731faf38f19d-plugins-config-volume\") pod \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") "
Sep 16 13:54:29.419930 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.419839    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg4xp\" (UniqueName: \"kubernetes.io/projected/5a5603e8-f66a-4b58-a7fb-731faf38f19d-kube-api-access-sg4xp\") pod \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\" (UID: \"5a5603e8-f66a-4b58-a7fb-731faf38f19d\") "
Sep 16 13:54:29.420054 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.420042    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mxzm\" (UniqueName: \"kubernetes.io/projected/02e506a3-6828-454e-bc09-2555469b3e39-kube-api-access-9mxzm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.420054 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.420054    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87g9g\" (UniqueName: \"kubernetes.io/projected/39055e3f-e328-4311-b0fc-8293fa1af697-kube-api-access-87g9g\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.420174 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.420063    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.420174 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.420072    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e506a3-6828-454e-bc09-2555469b3e39-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.420253 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.420173    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5603e8-f66a-4b58-a7fb-731faf38f19d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a5603e8-f66a-4b58-a7fb-731faf38f19d" (UID: "5a5603e8-f66a-4b58-a7fb-731faf38f19d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:54:29.421566 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.421523    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5603e8-f66a-4b58-a7fb-731faf38f19d-kube-api-access-sg4xp" (OuterVolumeSpecName: "kube-api-access-sg4xp") pod "5a5603e8-f66a-4b58-a7fb-731faf38f19d" (UID: "5a5603e8-f66a-4b58-a7fb-731faf38f19d"). InnerVolumeSpecName "kube-api-access-sg4xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:54:29.521368 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.521332    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg4xp\" (UniqueName: \"kubernetes.io/projected/5a5603e8-f66a-4b58-a7fb-731faf38f19d-kube-api-access-sg4xp\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.521368 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.521364    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5603e8-f66a-4b58-a7fb-731faf38f19d-plugins-config-volume\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:29.812930 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.812842    2556 generic.go:356] "Generic (PLEG): container finished" podID="5a5603e8-f66a-4b58-a7fb-731faf38f19d" containerID="355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95" exitCode=0
Sep 16 13:54:29.812930 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.812898    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"
Sep 16 13:54:29.813156 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.812927    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk" event={"ID":"5a5603e8-f66a-4b58-a7fb-731faf38f19d","Type":"ContainerDied","Data":"355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95"}
Sep 16 13:54:29.813156 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.812976    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk" event={"ID":"5a5603e8-f66a-4b58-a7fb-731faf38f19d","Type":"ContainerDied","Data":"0f68a55136b3ef05182f7bbe2ecce55833d036ce65c214850a6a7c4ccc572a4b"}
Sep 16 13:54:29.813156 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.813005    2556 scope.go:117] "RemoveContainer" containerID="355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95"
Sep 16 13:54:29.814375 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.814347    2556 generic.go:356] "Generic (PLEG): container finished" podID="02e506a3-6828-454e-bc09-2555469b3e39" containerID="513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab" exitCode=0
Sep 16 13:54:29.814514 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.814379    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv" event={"ID":"02e506a3-6828-454e-bc09-2555469b3e39","Type":"ContainerDied","Data":"513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab"}
Sep 16 13:54:29.814514 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.814423    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv" event={"ID":"02e506a3-6828-454e-bc09-2555469b3e39","Type":"ContainerDied","Data":"509297b8fd2608f271b3cf2db8bf5d396861c00fd3b1a94ad3e18efdb539e271"}
Sep 16 13:54:29.814514 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.814384    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"
Sep 16 13:54:29.815864 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.815834    2556 generic.go:356] "Generic (PLEG): container finished" podID="39055e3f-e328-4311-b0fc-8293fa1af697" containerID="bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d" exitCode=0
Sep 16 13:54:29.815994 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.815886    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"
Sep 16 13:54:29.815994 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.815916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5" event={"ID":"39055e3f-e328-4311-b0fc-8293fa1af697","Type":"ContainerDied","Data":"bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d"}
Sep 16 13:54:29.815994 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.815955    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5" event={"ID":"39055e3f-e328-4311-b0fc-8293fa1af697","Type":"ContainerDied","Data":"0ca45ef05ee4c5fe14832b0925fed7c426a92360abd776cc7c187556f86020f6"}
Sep 16 13:54:29.823542 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.823520    2556 scope.go:117] "RemoveContainer" containerID="355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95"
Sep 16 13:54:29.823966 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:54:29.823934    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95\": container with ID starting with 355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95 not found: ID does not exist" containerID="355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95"
Sep 16 13:54:29.824075 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.823974    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95"} err="failed to get container status \"355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95\": rpc error: code = NotFound desc = could not find container \"355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95\": container with ID starting with 355778c56c63fabb6a76e0d81beb5952c04ebaeb87f8949430daf3d10bcb3f95 not found: ID does not exist"
Sep 16 13:54:29.824075 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.823995    2556 scope.go:117] "RemoveContainer" containerID="513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab"
Sep 16 13:54:29.833032 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.833015    2556 scope.go:117] "RemoveContainer" containerID="513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab"
Sep 16 13:54:29.833277 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:54:29.833261    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab\": container with ID starting with 513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab not found: ID does not exist" containerID="513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab"
Sep 16 13:54:29.833347 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.833282    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab"} err="failed to get container status \"513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab\": rpc error: code = NotFound desc = could not find container \"513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab\": container with ID starting with 513910cd7eeab8ab45925dbff530877b82149fc6ea5c969b364dea13a693ddab not found: ID does not exist"
Sep 16 13:54:29.833347 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.833296    2556 scope.go:117] "RemoveContainer" containerID="bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d"
Sep 16 13:54:29.840754 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.840717    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"]
Sep 16 13:54:29.841044 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.841019    2556 scope.go:117] "RemoveContainer" containerID="bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d"
Sep 16 13:54:29.841354 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:54:29.841331    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d\": container with ID starting with bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d not found: ID does not exist" containerID="bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d"
Sep 16 13:54:29.841429 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.841357    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d"} err="failed to get container status \"bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d\": rpc error: code = NotFound desc = could not find container \"bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d\": container with ID starting with bb38b5d9edc85a601dc1131ad437de39d1584ad6bcb73b8cd49a7862fb07282d not found: ID does not exist"
Sep 16 13:54:29.842840 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.842822    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7e7bb4-85b4976754-jthsk"]
Sep 16 13:54:29.857926 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.857902    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"]
Sep 16 13:54:29.861522 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.861502    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7e7bb4-f88699b8f-5cvt5"]
Sep 16 13:54:29.870889 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.870868    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"]
Sep 16 13:54:29.876323 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:29.876303    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7e7bb4-d9888d55d-7m2lv"]
Sep 16 13:54:30.800504 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:30.800470    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02e506a3-6828-454e-bc09-2555469b3e39" path="/var/lib/kubelet/pods/02e506a3-6828-454e-bc09-2555469b3e39/volumes"
Sep 16 13:54:30.800874 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:30.800838    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39055e3f-e328-4311-b0fc-8293fa1af697" path="/var/lib/kubelet/pods/39055e3f-e328-4311-b0fc-8293fa1af697/volumes"
Sep 16 13:54:30.801156 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:30.801143    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a5603e8-f66a-4b58-a7fb-731faf38f19d" path="/var/lib/kubelet/pods/5a5603e8-f66a-4b58-a7fb-731faf38f19d/volumes"
Sep 16 13:54:32.034172 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.034141    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:54:32.351322 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.351228    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h" podUID="6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" containerName="main" containerID="cri-o://e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84" gracePeriod=45
Sep 16 13:54:32.596704 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.596682    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:54:32.750963 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.750938    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tmp-dir\") pod \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") "
Sep 16 13:54:32.751078 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.750970    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt44t\" (UniqueName: \"kubernetes.io/projected/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-kube-api-access-mt44t\") pod \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") "
Sep 16 13:54:32.751078 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751015    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-dshm\") pod \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") "
Sep 16 13:54:32.751078 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751034    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-home\") pod \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") "
Sep 16 13:54:32.751078 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751075    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tls-certs\") pod \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") "
Sep 16 13:54:32.751289 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751116    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-model-cache\") pod \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\" (UID: \"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a\") "
Sep 16 13:54:32.751289 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751192    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" (UID: "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:32.751397 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751376    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-home" (OuterVolumeSpecName: "home") pod "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" (UID: "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:32.751458 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751400    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" (UID: "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:32.751458 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.751408    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:32.753383 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.753357    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-dshm" (OuterVolumeSpecName: "dshm") pod "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" (UID: "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:32.753478 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.753422    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" (UID: "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:54:32.753525 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.753499    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-kube-api-access-mt44t" (OuterVolumeSpecName: "kube-api-access-mt44t") pod "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" (UID: "6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a"). InnerVolumeSpecName "kube-api-access-mt44t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:54:32.837014 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.836985    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" containerID="e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84" exitCode=0
Sep 16 13:54:32.837123 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.837019    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"
Sep 16 13:54:32.837123 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.837075    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h" event={"ID":"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a","Type":"ContainerDied","Data":"e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84"}
Sep 16 13:54:32.837123 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.837118    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h" event={"ID":"6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a","Type":"ContainerDied","Data":"d710e3e70173d8a50cdfa109fb8f2d67f12e9e930d8d6967300d42da903f8022"}
Sep 16 13:54:32.837292 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.837142    2556 scope.go:117] "RemoveContainer" containerID="e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84"
Sep 16 13:54:32.845214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.845198    2556 scope.go:117] "RemoveContainer" containerID="e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84"
Sep 16 13:54:32.845471 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:54:32.845451    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84\": container with ID starting with e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84 not found: ID does not exist" containerID="e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84"
Sep 16 13:54:32.845541 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.845482    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84"} err="failed to get container status \"e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84\": rpc error: code = NotFound desc = could not find container \"e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84\": container with ID starting with e339446c81cfa24eab9af51a749c9e45bc58ac06b6522861036a6a1629e68b84 not found: ID does not exist"
Sep 16 13:54:32.851826 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.851803    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"]
Sep 16 13:54:32.851906 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.851878    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:32.851906 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.851893    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt44t\" (UniqueName: \"kubernetes.io/projected/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-kube-api-access-mt44t\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:32.851906 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.851903    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:32.852016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.851913    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:32.852016 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.851920    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:32.855154 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:32.855134    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7e7bb4/auth-test-model-7e7bb4-kserve-64cfbdf7b7-jt92h"]
Sep 16 13:54:33.323560 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.323504    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4" podUID="ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" containerName="main" containerID="cri-o://0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273" gracePeriod=45
Sep 16 13:54:33.569476 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.569452    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:54:33.658812 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.658789    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-home\") pod \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") "
Sep 16 13:54:33.658955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.658817    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tls-certs\") pod \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") "
Sep 16 13:54:33.658955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.658836    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9x8p\" (UniqueName: \"kubernetes.io/projected/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-kube-api-access-h9x8p\") pod \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") "
Sep 16 13:54:33.658955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.658862    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-dshm\") pod \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") "
Sep 16 13:54:33.658955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.658882    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-model-cache\") pod \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") "
Sep 16 13:54:33.658955 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.658921    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tmp-dir\") pod \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\" (UID: \"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f\") "
Sep 16 13:54:33.659214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.659105    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-home" (OuterVolumeSpecName: "home") pod "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" (UID: "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:33.659214 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.659121    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" (UID: "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:33.659290 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.659254    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" (UID: "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:33.661029 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.661008    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-dshm" (OuterVolumeSpecName: "dshm") pod "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" (UID: "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:54:33.661118 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.661023    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-kube-api-access-h9x8p" (OuterVolumeSpecName: "kube-api-access-h9x8p") pod "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" (UID: "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f"). InnerVolumeSpecName "kube-api-access-h9x8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:54:33.661118 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.661055    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" (UID: "ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:54:33.760128 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.760105    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-home\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:33.760128 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.760124    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tls-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:33.760255 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.760134    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9x8p\" (UniqueName: \"kubernetes.io/projected/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-kube-api-access-h9x8p\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:33.760255 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.760143    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-dshm\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:33.760255 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.760151    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-model-cache\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:33.760255 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.760159    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f-tmp-dir\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:33.842858 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.842832    2556 generic.go:356] "Generic (PLEG): container finished" podID="ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" containerID="0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273" exitCode=0
Sep 16 13:54:33.842965 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.842885    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"
Sep 16 13:54:33.842965 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.842919    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4" event={"ID":"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f","Type":"ContainerDied","Data":"0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273"}
Sep 16 13:54:33.843075 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.842978    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4" event={"ID":"ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f","Type":"ContainerDied","Data":"8b5ced14a6b2cde2d66486502b68e7c17db05bd2c061b9a41bffded2fab27788"}
Sep 16 13:54:33.843075 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.843004    2556 scope.go:117] "RemoveContainer" containerID="0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273"
Sep 16 13:54:33.851972 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.851958    2556 scope.go:117] "RemoveContainer" containerID="0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273"
Sep 16 13:54:33.852210 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:54:33.852191    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273\": container with ID starting with 0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273 not found: ID does not exist" containerID="0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273"
Sep 16 13:54:33.852294 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.852217    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273"} err="failed to get container status \"0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273\": rpc error: code = NotFound desc = could not find container \"0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273\": container with ID starting with 0555fd63289ded5d6e4c91c4079ba8351e5a85577da1e1e676065dfcd677a273 not found: ID does not exist"
Sep 16 13:54:33.863763 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.863743    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"]
Sep 16 13:54:33.869019 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:33.868998    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8c3cdf/auth-test-model-8c3cdf-kserve-66b587b4c5-kzvm4"]
Sep 16 13:54:34.800192 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:34.800151    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" path="/var/lib/kubelet/pods/6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a/volumes"
Sep 16 13:54:34.800744 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:34.800723    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" path="/var/lib/kubelet/pods/ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f/volumes"
Sep 16 13:54:46.590765 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.590731    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"]
Sep 16 13:54:46.591198 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.591017    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" podUID="990ae36f-bb87-4666-9716-d720d4bcc719" containerName="maas-api" containerID="cri-o://d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b" gracePeriod=30
Sep 16 13:54:46.826422 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.826394    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:54:46.904378 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.904343    2556 generic.go:356] "Generic (PLEG): container finished" podID="990ae36f-bb87-4666-9716-d720d4bcc719" containerID="d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b" exitCode=0
Sep 16 13:54:46.904508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.904384    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"
Sep 16 13:54:46.904508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.904431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" event={"ID":"990ae36f-bb87-4666-9716-d720d4bcc719","Type":"ContainerDied","Data":"d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b"}
Sep 16 13:54:46.904508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.904480    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz" event={"ID":"990ae36f-bb87-4666-9716-d720d4bcc719","Type":"ContainerDied","Data":"16ae868a6765c0c62f56edfea323c58d944096404fe620d686ce70c3b1f6976c"}
Sep 16 13:54:46.904508 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.904504    2556 scope.go:117] "RemoveContainer" containerID="d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b"
Sep 16 13:54:46.912532 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.912514    2556 scope.go:117] "RemoveContainer" containerID="d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b"
Sep 16 13:54:46.912820 ip-10-0-143-82 kubenswrapper[2556]: E0916 13:54:46.912801    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b\": container with ID starting with d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b not found: ID does not exist" containerID="d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b"
Sep 16 13:54:46.912895 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.912827    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b"} err="failed to get container status \"d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b\": rpc error: code = NotFound desc = could not find container \"d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b\": container with ID starting with d50951e923221a174238cfc73ffa375c42e3ddbb8057137d859b75feecb2908b not found: ID does not exist"
Sep 16 13:54:46.958567 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.958544    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6wg2\" (UniqueName: \"kubernetes.io/projected/990ae36f-bb87-4666-9716-d720d4bcc719-kube-api-access-d6wg2\") pod \"990ae36f-bb87-4666-9716-d720d4bcc719\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") "
Sep 16 13:54:46.958667 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.958585    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990ae36f-bb87-4666-9716-d720d4bcc719-openshift-service-ca\") pod \"990ae36f-bb87-4666-9716-d720d4bcc719\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") "
Sep 16 13:54:46.958667 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.958631    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-metrics-certs\") pod \"990ae36f-bb87-4666-9716-d720d4bcc719\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") "
Sep 16 13:54:46.958667 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.958662    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls\") pod \"990ae36f-bb87-4666-9716-d720d4bcc719\" (UID: \"990ae36f-bb87-4666-9716-d720d4bcc719\") "
Sep 16 13:54:46.958982 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.958958    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/990ae36f-bb87-4666-9716-d720d4bcc719-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "990ae36f-bb87-4666-9716-d720d4bcc719" (UID: "990ae36f-bb87-4666-9716-d720d4bcc719"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:54:46.960575 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.960550    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/990ae36f-bb87-4666-9716-d720d4bcc719-kube-api-access-d6wg2" (OuterVolumeSpecName: "kube-api-access-d6wg2") pod "990ae36f-bb87-4666-9716-d720d4bcc719" (UID: "990ae36f-bb87-4666-9716-d720d4bcc719"). InnerVolumeSpecName "kube-api-access-d6wg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:54:46.960727 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.960587    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "990ae36f-bb87-4666-9716-d720d4bcc719" (UID: "990ae36f-bb87-4666-9716-d720d4bcc719"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:54:46.960778 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:46.960722    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "990ae36f-bb87-4666-9716-d720d4bcc719" (UID: "990ae36f-bb87-4666-9716-d720d4bcc719"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:54:47.059954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:47.059927    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-metrics-certs\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:47.059954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:47.059950    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990ae36f-bb87-4666-9716-d720d4bcc719-maas-api-tls\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:47.059954 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:47.059959    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6wg2\" (UniqueName: \"kubernetes.io/projected/990ae36f-bb87-4666-9716-d720d4bcc719-kube-api-access-d6wg2\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:47.060192 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:47.059968    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990ae36f-bb87-4666-9716-d720d4bcc719-openshift-service-ca\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 13:54:47.227387 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:47.227365    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"]
Sep 16 13:54:47.230078 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:47.230055    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-8c3cdf-6cc7f6dcd7-72vdz"]
Sep 16 13:54:48.799887 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:48.799854    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="990ae36f-bb87-4666-9716-d720d4bcc719" path="/var/lib/kubelet/pods/990ae36f-bb87-4666-9716-d720d4bcc719/volumes"
Sep 16 13:54:49.443570 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:54:49.443537    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tfhpd"]
Sep 16 13:58:13.015867 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:58:13.015841    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 13:58:13.026433 ip-10-0-143-82 kubenswrapper[2556]: I0916 13:58:13.026410    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 14:00:00.190825 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.190790    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"]
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191122    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" containerName="main"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191134    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" containerName="main"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191146    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" containerName="main"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191151    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" containerName="main"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191160    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b65873d1-c729-4652-8d5c-44f202142776" containerName="maas-api"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191167    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b65873d1-c729-4652-8d5c-44f202142776" containerName="maas-api"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191173    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5944edf6-d977-4a86-b827-fe8f2a01382d" containerName="cleanup"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191178    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5944edf6-d977-4a86-b827-fe8f2a01382d" containerName="cleanup"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191187    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5603e8-f66a-4b58-a7fb-731faf38f19d" containerName="payload-pre-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191192    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5603e8-f66a-4b58-a7fb-731faf38f19d" containerName="payload-pre-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191200    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f90b708c-5543-4462-b90e-110deb34cb97" containerName="cleanup"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191205    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f90b708c-5543-4462-b90e-110deb34cb97" containerName="cleanup"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191214    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="990ae36f-bb87-4666-9716-d720d4bcc719" containerName="maas-api"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191219    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="990ae36f-bb87-4666-9716-d720d4bcc719" containerName="maas-api"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191226    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e506a3-6828-454e-bc09-2555469b3e39" containerName="maas-api"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191231    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e506a3-6828-454e-bc09-2555469b3e39" containerName="maas-api"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191237    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01d67d28-55ed-443c-9aab-43476531e2d7" containerName="payload-pre-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191242    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="01d67d28-55ed-443c-9aab-43476531e2d7" containerName="payload-pre-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191248    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66fc331d-fd62-474d-8427-215860f0a583" containerName="payload-pre-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191253    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="66fc331d-fd62-474d-8427-215860f0a583" containerName="payload-pre-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191260    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39055e3f-e328-4311-b0fc-8293fa1af697" containerName="payload-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191265    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="39055e3f-e328-4311-b0fc-8293fa1af697" containerName="payload-processing"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191271    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70051f72-b766-4b03-ae3c-a36b055cb25b" containerName="cleanup"
Sep 16 14:00:00.191285 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191276    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="70051f72-b766-4b03-ae3c-a36b055cb25b" containerName="cleanup"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191324    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5603e8-f66a-4b58-a7fb-731faf38f19d" containerName="payload-pre-processing"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191331    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff02c1e2-cbd9-4719-8026-ff2f4d33ea3f" containerName="main"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191337    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e506a3-6828-454e-bc09-2555469b3e39" containerName="maas-api"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191344    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="39055e3f-e328-4311-b0fc-8293fa1af697" containerName="payload-processing"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191349    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="66fc331d-fd62-474d-8427-215860f0a583" containerName="payload-pre-processing"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191355    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cbbdd0e-78e4-44ab-92f2-aa1e3d32c39a" containerName="main"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191360    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b65873d1-c729-4652-8d5c-44f202142776" containerName="maas-api"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191364    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="990ae36f-bb87-4666-9716-d720d4bcc719" containerName="maas-api"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191370    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="70051f72-b766-4b03-ae3c-a36b055cb25b" containerName="cleanup"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191377    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5944edf6-d977-4a86-b827-fe8f2a01382d" containerName="cleanup"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191383    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="01d67d28-55ed-443c-9aab-43476531e2d7" containerName="payload-pre-processing"
Sep 16 14:00:00.192053 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.191388    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f90b708c-5543-4462-b90e-110deb34cb97" containerName="cleanup"
Sep 16 14:00:00.194209 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.194192    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"]
Sep 16 14:00:00.194340 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.194319    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:00.197344 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.197328    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:00.200331 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.200314    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nvp5m\""
Sep 16 14:00:00.223884 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.223861    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"]
Sep 16 14:00:00.226366 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.226340    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"]
Sep 16 14:00:00.295248 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.295223    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zl7f\" (UniqueName: \"kubernetes.io/projected/a62f266d-4b71-4d57-ad45-87539f4b06c0-kube-api-access-9zl7f\") pod \"maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h\" (UID: \"a62f266d-4b71-4d57-ad45-87539f4b06c0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:00.295344 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.295265    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6znds\" (UniqueName: \"kubernetes.io/projected/9bb02b0c-f791-4149-9c11-da0f841f652c-kube-api-access-6znds\") pod \"maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6\" (UID: \"9bb02b0c-f791-4149-9c11-da0f841f652c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:00.395944 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.395918    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zl7f\" (UniqueName: \"kubernetes.io/projected/a62f266d-4b71-4d57-ad45-87539f4b06c0-kube-api-access-9zl7f\") pod \"maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h\" (UID: \"a62f266d-4b71-4d57-ad45-87539f4b06c0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:00.396036 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.395962    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6znds\" (UniqueName: \"kubernetes.io/projected/9bb02b0c-f791-4149-9c11-da0f841f652c-kube-api-access-6znds\") pod \"maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6\" (UID: \"9bb02b0c-f791-4149-9c11-da0f841f652c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:00.405911 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.405889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zl7f\" (UniqueName: \"kubernetes.io/projected/a62f266d-4b71-4d57-ad45-87539f4b06c0-kube-api-access-9zl7f\") pod \"maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h\" (UID: \"a62f266d-4b71-4d57-ad45-87539f4b06c0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:00.406010 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.405989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6znds\" (UniqueName: \"kubernetes.io/projected/9bb02b0c-f791-4149-9c11-da0f841f652c-kube-api-access-6znds\") pod \"maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6\" (UID: \"9bb02b0c-f791-4149-9c11-da0f841f652c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:00.505045 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.504975    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:00.510752 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.510732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:00.790284 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.790206    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"]
Sep 16 14:00:00.793286 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:00.793164    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"]
Sep 16 14:00:00.793865 ip-10-0-143-82 kubenswrapper[2556]: W0916 14:00:00.793833    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62f266d_4b71_4d57_ad45_87539f4b06c0.slice/crio-36c64e8c8446bbb77ce26278cbadaa4b004eeea744e2ce4b722113862e1378fd WatchSource:0}: Error finding container 36c64e8c8446bbb77ce26278cbadaa4b004eeea744e2ce4b722113862e1378fd: Status 404 returned error can't find the container with id 36c64e8c8446bbb77ce26278cbadaa4b004eeea744e2ce4b722113862e1378fd
Sep 16 14:00:00.795410 ip-10-0-143-82 kubenswrapper[2556]: W0916 14:00:00.795386    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb02b0c_f791_4149_9c11_da0f841f652c.slice/crio-7146a762587fb734b6131687554f678dd513ae80afcfc4a5e1c28b13dd0e5844 WatchSource:0}: Error finding container 7146a762587fb734b6131687554f678dd513ae80afcfc4a5e1c28b13dd0e5844: Status 404 returned error can't find the container with id 7146a762587fb734b6131687554f678dd513ae80afcfc4a5e1c28b13dd0e5844
Sep 16 14:00:01.350957 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:01.350908    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6" event={"ID":"9bb02b0c-f791-4149-9c11-da0f841f652c","Type":"ContainerStarted","Data":"d61f88280549f2a1791bdaa027a3251d599639c85938d6ac24d32c29e3e55360"}
Sep 16 14:00:01.351407 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:01.351008    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6" event={"ID":"9bb02b0c-f791-4149-9c11-da0f841f652c","Type":"ContainerStarted","Data":"7146a762587fb734b6131687554f678dd513ae80afcfc4a5e1c28b13dd0e5844"}
Sep 16 14:00:01.352467 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:01.352440    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h" event={"ID":"a62f266d-4b71-4d57-ad45-87539f4b06c0","Type":"ContainerStarted","Data":"f8bb8cce47b795786a39cb6579fff94cd93698184f5d782904835e7c90758590"}
Sep 16 14:00:01.352571 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:01.352488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h" event={"ID":"a62f266d-4b71-4d57-ad45-87539f4b06c0","Type":"ContainerStarted","Data":"36c64e8c8446bbb77ce26278cbadaa4b004eeea744e2ce4b722113862e1378fd"}
Sep 16 14:00:01.369208 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:01.369158    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6" podStartSLOduration=1.369139751 podStartE2EDuration="1.369139751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:00:01.367901153 +0000 UTC m=+3109.044615589" watchObservedRunningTime="2026-09-16 14:00:01.369139751 +0000 UTC m=+3109.045854186"
Sep 16 14:00:01.386688 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:01.386651    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h" podStartSLOduration=1.386638622 podStartE2EDuration="1.386638622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:00:01.385571643 +0000 UTC m=+3109.062286077" watchObservedRunningTime="2026-09-16 14:00:01.386638622 +0000 UTC m=+3109.063353070"
Sep 16 14:00:02.357983 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:02.357948    2556 generic.go:356] "Generic (PLEG): container finished" podID="a62f266d-4b71-4d57-ad45-87539f4b06c0" containerID="f8bb8cce47b795786a39cb6579fff94cd93698184f5d782904835e7c90758590" exitCode=0
Sep 16 14:00:02.358398 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:02.358032    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h" event={"ID":"a62f266d-4b71-4d57-ad45-87539f4b06c0","Type":"ContainerDied","Data":"f8bb8cce47b795786a39cb6579fff94cd93698184f5d782904835e7c90758590"}
Sep 16 14:00:02.359573 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:02.359549    2556 generic.go:356] "Generic (PLEG): container finished" podID="9bb02b0c-f791-4149-9c11-da0f841f652c" containerID="d61f88280549f2a1791bdaa027a3251d599639c85938d6ac24d32c29e3e55360" exitCode=0
Sep 16 14:00:02.359698 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:02.359591    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6" event={"ID":"9bb02b0c-f791-4149-9c11-da0f841f652c","Type":"ContainerDied","Data":"d61f88280549f2a1791bdaa027a3251d599639c85938d6ac24d32c29e3e55360"}
Sep 16 14:00:03.502572 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.502545    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:03.528997 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.528974    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:03.623654 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.623623    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zl7f\" (UniqueName: \"kubernetes.io/projected/a62f266d-4b71-4d57-ad45-87539f4b06c0-kube-api-access-9zl7f\") pod \"a62f266d-4b71-4d57-ad45-87539f4b06c0\" (UID: \"a62f266d-4b71-4d57-ad45-87539f4b06c0\") "
Sep 16 14:00:03.623835 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.623691    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6znds\" (UniqueName: \"kubernetes.io/projected/9bb02b0c-f791-4149-9c11-da0f841f652c-kube-api-access-6znds\") pod \"9bb02b0c-f791-4149-9c11-da0f841f652c\" (UID: \"9bb02b0c-f791-4149-9c11-da0f841f652c\") "
Sep 16 14:00:03.625715 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.625687    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb02b0c-f791-4149-9c11-da0f841f652c-kube-api-access-6znds" (OuterVolumeSpecName: "kube-api-access-6znds") pod "9bb02b0c-f791-4149-9c11-da0f841f652c" (UID: "9bb02b0c-f791-4149-9c11-da0f841f652c"). InnerVolumeSpecName "kube-api-access-6znds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:00:03.625715 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.625701    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a62f266d-4b71-4d57-ad45-87539f4b06c0-kube-api-access-9zl7f" (OuterVolumeSpecName: "kube-api-access-9zl7f") pod "a62f266d-4b71-4d57-ad45-87539f4b06c0" (UID: "a62f266d-4b71-4d57-ad45-87539f4b06c0"). InnerVolumeSpecName "kube-api-access-9zl7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:00:03.724549 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.724512    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zl7f\" (UniqueName: \"kubernetes.io/projected/a62f266d-4b71-4d57-ad45-87539f4b06c0-kube-api-access-9zl7f\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 14:00:03.724549 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:03.724547    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6znds\" (UniqueName: \"kubernetes.io/projected/9bb02b0c-f791-4149-9c11-da0f841f652c-kube-api-access-6znds\") on node \"ip-10-0-143-82.ec2.internal\" DevicePath \"\""
Sep 16 14:00:04.370011 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:04.369979    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6"
Sep 16 14:00:04.370202 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:04.370033    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6" event={"ID":"9bb02b0c-f791-4149-9c11-da0f841f652c","Type":"ContainerDied","Data":"7146a762587fb734b6131687554f678dd513ae80afcfc4a5e1c28b13dd0e5844"}
Sep 16 14:00:04.370202 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:04.370088    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7146a762587fb734b6131687554f678dd513ae80afcfc4a5e1c28b13dd0e5844"
Sep 16 14:00:04.371299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:04.371277    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h" event={"ID":"a62f266d-4b71-4d57-ad45-87539f4b06c0","Type":"ContainerDied","Data":"36c64e8c8446bbb77ce26278cbadaa4b004eeea744e2ce4b722113862e1378fd"}
Sep 16 14:00:04.371424 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:04.371308    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h"
Sep 16 14:00:04.371424 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:00:04.371310    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36c64e8c8446bbb77ce26278cbadaa4b004eeea744e2ce4b722113862e1378fd"
Sep 16 14:03:04.847067 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:04.847040    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85fb6fc7f5-ddcxv_dc6d7472-ce26-4447-834b-6950cb9b43bb/authorino/0.log"
Sep 16 14:03:11.056460 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:11.056425    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694ff5d95c-p4mhm_183d736d-9ec9-4501-8d34-62c2d5bd81d5/copy-manifests/0.log"
Sep 16 14:03:11.064032 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:11.064000    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694ff5d95c-p4mhm_183d736d-9ec9-4501-8d34-62c2d5bd81d5/manager/0.log"
Sep 16 14:03:11.818224 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:11.818192    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-inf-b-c71ad7-55bbb7489b-kjdp8_cfad9548-8033-44fc-892f-e4ec78c9a61d/maas-api/0.log"
Sep 16 14:03:12.776532 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:12.776506    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-a-48adf5-29826120-zws6h_a62f266d-4b71-4d57-ad45-87539f4b06c0/cleanup/0.log"
Sep 16 14:03:12.912610 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:12.912568    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-b-c71ad7-29826090-z2m56_cbbcb059-b8cc-4cc5-a21f-f2810d051af3/cleanup/0.log"
Sep 16 14:03:13.045733 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:13.045660    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 14:03:13.055571 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:13.055551    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 14:03:13.733123 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:13.733095    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826105-lf4hp_5944edf6-d977-4a86-b827-fe8f2a01382d/cleanup/0.log"
Sep 16 14:03:13.849902 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:13.849876    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-worker-w4-f27ea8-29826120-gkxp6_9bb02b0c-f791-4149-9c11-da0f841f652c/cleanup/0.log"
Sep 16 14:03:14.845477 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:14.845433    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85fb6fc7f5-ddcxv_dc6d7472-ce26-4447-834b-6950cb9b43bb/authorino/0.log"
Sep 16 14:03:15.217402 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:15.217380    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9npcm_b6631883-c889-456c-864d-996e1b6da645/kuadrant-console-plugin/0.log"
Sep 16 14:03:15.447269 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:15.447236    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-tfhpd_d517954a-0f70-4689-bf72-edc01d46b1b1/limitador/0.log"
Sep 16 14:03:16.210266 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:16.210240    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6445dbdc5b-5z7fn_cd97e5ad-545f-4bee-b1ab-d9eca1b81626/kube-auth-proxy/0.log"
Sep 16 14:03:16.799889 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:16.799833    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff_f2aea88a-9474-43bc-a8df-d0fe082cf3a8/payload-pre-processing/0.log"
Sep 16 14:03:16.925191 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:16.925159    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49_ecb26fad-12f8-49f4-8d6e-2e52d3cc8139/payload-pre-processing/0.log"
Sep 16 14:03:17.408021 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:17.407999    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9_6e767317-491f-4b74-b6ec-64e488ea4c0a/payload-processing/0.log"
Sep 16 14:03:17.775237 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:17.775164    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-675764b9db-82s2n_01e0e7b1-ef76-4103-99cf-588da4736933/router/0.log"
Sep 16 14:03:18.516283 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:18.516251    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lr2mg_7767e174-64d3-48dd-a0f0-4efe5e9cd660/main/0.log"
Sep 16 14:03:18.863999 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:18.863918    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8fjkp_5d117f9b-9849-4bbe-b6c6-25b0b11a9c32/main/0.log"
Sep 16 14:03:24.549805 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.549762    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hrntr/must-gather-hnm94"]
Sep 16 14:03:24.550299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.550114    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb02b0c-f791-4149-9c11-da0f841f652c" containerName="cleanup"
Sep 16 14:03:24.550299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.550127    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb02b0c-f791-4149-9c11-da0f841f652c" containerName="cleanup"
Sep 16 14:03:24.550299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.550143    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62f266d-4b71-4d57-ad45-87539f4b06c0" containerName="cleanup"
Sep 16 14:03:24.550299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.550149    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62f266d-4b71-4d57-ad45-87539f4b06c0" containerName="cleanup"
Sep 16 14:03:24.550299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.550197    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb02b0c-f791-4149-9c11-da0f841f652c" containerName="cleanup"
Sep 16 14:03:24.550299 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.550206    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a62f266d-4b71-4d57-ad45-87539f4b06c0" containerName="cleanup"
Sep 16 14:03:24.553243 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.553226    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.556239 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.556216    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hrntr\"/\"openshift-service-ca.crt\""
Sep 16 14:03:24.558445 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.558422    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hrntr\"/\"kube-root-ca.crt\""
Sep 16 14:03:24.574055 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.574032    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hrntr/must-gather-hnm94"]
Sep 16 14:03:24.691671 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.691648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8cm5\" (UniqueName: \"kubernetes.io/projected/8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c-kube-api-access-b8cm5\") pod \"must-gather-hnm94\" (UID: \"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c\") " pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.691782 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.691691    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c-must-gather-output\") pod \"must-gather-hnm94\" (UID: \"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c\") " pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.792142 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.792113    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8cm5\" (UniqueName: \"kubernetes.io/projected/8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c-kube-api-access-b8cm5\") pod \"must-gather-hnm94\" (UID: \"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c\") " pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.792257 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.792169    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c-must-gather-output\") pod \"must-gather-hnm94\" (UID: \"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c\") " pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.792504 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.792485    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c-must-gather-output\") pod \"must-gather-hnm94\" (UID: \"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c\") " pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.801034 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.800979    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8cm5\" (UniqueName: \"kubernetes.io/projected/8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c-kube-api-access-b8cm5\") pod \"must-gather-hnm94\" (UID: \"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c\") " pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:24.862071 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:24.862034    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hrntr/must-gather-hnm94"
Sep 16 14:03:25.132753 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:25.132729    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hrntr/must-gather-hnm94"]
Sep 16 14:03:25.135108 ip-10-0-143-82 kubenswrapper[2556]: W0916 14:03:25.135074    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8aca3a35_3592_4d0f_98f0_4f5f2bb3fe5c.slice/crio-153c77d2747b4d42dd2897a8ed443d003d697078933b30b1dc16a0f54c5dbccb WatchSource:0}: Error finding container 153c77d2747b4d42dd2897a8ed443d003d697078933b30b1dc16a0f54c5dbccb: Status 404 returned error can't find the container with id 153c77d2747b4d42dd2897a8ed443d003d697078933b30b1dc16a0f54c5dbccb
Sep 16 14:03:25.136829 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:25.136811    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:03:25.299814 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:25.299791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrntr/must-gather-hnm94" event={"ID":"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c","Type":"ContainerStarted","Data":"153c77d2747b4d42dd2897a8ed443d003d697078933b30b1dc16a0f54c5dbccb"}
Sep 16 14:03:26.306930 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:26.306899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrntr/must-gather-hnm94" event={"ID":"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c","Type":"ContainerStarted","Data":"6e7ce262faa8f6e34635d5db6dedf9d61f0ba45fa0ce5dfd96ef4dfb3c4fcd81"}
Sep 16 14:03:27.314370 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:27.314327    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrntr/must-gather-hnm94" event={"ID":"8aca3a35-3592-4d0f-98f0-4f5f2bb3fe5c","Type":"ContainerStarted","Data":"1eba717760fe15898380aabe4cb449931f01595f5bc7c399303278a88bb0ea7f"}
Sep 16 14:03:27.334014 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:27.333959    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hrntr/must-gather-hnm94" podStartSLOduration=2.353550525 podStartE2EDuration="3.333942488s" totalImagesPullingTime="980.391963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:03:27.33270955 +0000 UTC m=+3315.009424014" watchObservedRunningTime="2026-09-16 14:03:27.333942488 +0000 UTC m=+3315.010656923"
Sep 16 14:03:27.641021 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:27.640976    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2n6bp_694264fb-023d-4197-b834-514ec068625c/global-pull-secret-syncer/0.log"
Sep 16 14:03:27.816312 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:27.816279    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nj4sb_3229d434-09bf-404f-b75a-bbb01395351e/konnectivity-agent/0.log"
Sep 16 14:03:27.919931 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:27.919857    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-82.ec2.internal_32a26a83e8371e1e188502dccce97eac/haproxy/0.log"
Sep 16 14:03:32.094929 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:32.094671    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85fb6fc7f5-ddcxv_dc6d7472-ce26-4447-834b-6950cb9b43bb/authorino/0.log"
Sep 16 14:03:32.194850 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:32.194610    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9npcm_b6631883-c889-456c-864d-996e1b6da645/kuadrant-console-plugin/0.log"
Sep 16 14:03:32.321742 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:32.321714    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-tfhpd_d517954a-0f70-4689-bf72-edc01d46b1b1/limitador/0.log"
Sep 16 14:03:34.212226 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:34.212177    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-7bv2q_e1502655-4118-4947-86c7-0040ae5a3791/cluster-monitoring-operator/0.log"
Sep 16 14:03:34.653677 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:34.653611    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t9j4r_b41f34de-bcd5-418c-893d-ba16e97f2c99/node-exporter/0.log"
Sep 16 14:03:34.683773 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:34.683739    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t9j4r_b41f34de-bcd5-418c-893d-ba16e97f2c99/kube-rbac-proxy/0.log"
Sep 16 14:03:34.714795 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:34.714771    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t9j4r_b41f34de-bcd5-418c-893d-ba16e97f2c99/init-textfile/0.log"
Sep 16 14:03:37.467232 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:37.467197    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/2.log"
Sep 16 14:03:37.472096 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:37.472066    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-878hd_91d784c7-fff8-4c82-9f66-70abca2656e9/console-operator/3.log"
Sep 16 14:03:38.104428 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.104382    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"]
Sep 16 14:03:38.110250 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.110224    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.113448 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.113425    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hrntr\"/\"default-dockercfg-pll6d\""
Sep 16 14:03:38.123293 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.123267    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"]
Sep 16 14:03:38.239761 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.239725    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-proc\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.240016 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.239995    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-podres\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.240161 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.240142    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-lib-modules\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.240290 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.240278    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-sys\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.240422 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.240409    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgpb8\" (UniqueName: \"kubernetes.io/projected/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-kube-api-access-xgpb8\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.341887 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.341853    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-sys\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342076 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.341900    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgpb8\" (UniqueName: \"kubernetes.io/projected/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-kube-api-access-xgpb8\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342076 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.341970    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-sys\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342076 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.342013    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-proc\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342076 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.342031    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-podres\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342076 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.342051    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-lib-modules\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342351 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.342166    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-lib-modules\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342351 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.342183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-podres\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.342351 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.342280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-proc\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.350237 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.350201    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgpb8\" (UniqueName: \"kubernetes.io/projected/f7af8a4a-643b-407b-a7ad-4a9c9c22e1db-kube-api-access-xgpb8\") pod \"perf-node-gather-daemonset-6bmqp\" (UID: \"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db\") " pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.422905 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.422661    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:38.731057 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.728420    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"]
Sep 16 14:03:38.732089 ip-10-0-143-82 kubenswrapper[2556]: W0916 14:03:38.732050    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7af8a4a_643b_407b_a7ad_4a9c9c22e1db.slice/crio-7e813faf084d2ae1599e4a0d7296e59232d56699a9797f2a69c69426af1f955e WatchSource:0}: Error finding container 7e813faf084d2ae1599e4a0d7296e59232d56699a9797f2a69c69426af1f955e: Status 404 returned error can't find the container with id 7e813faf084d2ae1599e4a0d7296e59232d56699a9797f2a69c69426af1f955e
Sep 16 14:03:38.819317 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:38.819283    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-jczcv_f1dcc4c6-5d6e-42d9-8eca-7ffc980e19cd/volume-data-source-validator/0.log"
Sep 16 14:03:39.395223 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:39.395177    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp" event={"ID":"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db","Type":"ContainerStarted","Data":"623558a8c0809f586fab2279d6ecdf17723977adf60a3af0ea5b78b7127d4d1e"}
Sep 16 14:03:39.395416 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:39.395254    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp" event={"ID":"f7af8a4a-643b-407b-a7ad-4a9c9c22e1db","Type":"ContainerStarted","Data":"7e813faf084d2ae1599e4a0d7296e59232d56699a9797f2a69c69426af1f955e"}
Sep 16 14:03:39.395568 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:39.395546    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:39.422202 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:39.422141    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp" podStartSLOduration=1.42212091 podStartE2EDuration="1.42212091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:03:39.420709614 +0000 UTC m=+3327.097424050" watchObservedRunningTime="2026-09-16 14:03:39.42212091 +0000 UTC m=+3327.098835346"
Sep 16 14:03:39.913404 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:39.913371    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hn46r_6c759f5e-28c4-4176-9f32-b75b43efc524/dns/0.log"
Sep 16 14:03:39.932798 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:39.932771    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hn46r_6c759f5e-28c4-4176-9f32-b75b43efc524/kube-rbac-proxy/0.log"
Sep 16 14:03:40.095643 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:40.095577    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-td4wf_0a4bdde9-8ef0-4089-9fdf-9414452b6494/dns-node-resolver/0.log"
Sep 16 14:03:40.778531 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:40.778502    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rfr7l_3a2b4109-e4a9-467b-8ec7-a71ca968b738/node-ca/0.log"
Sep 16 14:03:42.307529 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:42.307505    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6445dbdc5b-5z7fn_cd97e5ad-545f-4bee-b1ab-d9eca1b81626/kube-auth-proxy/0.log"
Sep 16 14:03:42.448273 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:42.448244    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff_f2aea88a-9474-43bc-a8df-d0fe082cf3a8/payload-pre-processing/0.log"
Sep 16 14:03:42.478104 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:42.478075    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49_ecb26fad-12f8-49f4-8d6e-2e52d3cc8139/payload-pre-processing/0.log"
Sep 16 14:03:42.590555 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:42.590486    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9_6e767317-491f-4b74-b6ec-64e488ea4c0a/payload-processing/0.log"
Sep 16 14:03:42.677839 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:42.677810    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-675764b9db-82s2n_01e0e7b1-ef76-4103-99cf-588da4736933/router/0.log"
Sep 16 14:03:43.625235 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:43.625202    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dlp96_8455cf6b-b3c9-4c05-b62d-613ed31ba4f0/serve-healthcheck-canary/0.log"
Sep 16 14:03:44.270953 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:44.270928    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-v86g5_1219ffb1-73d9-4ea2-a992-2d379213dc9f/insights-operator/0.log"
Sep 16 14:03:44.272033 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:44.272017    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-v86g5_1219ffb1-73d9-4ea2-a992-2d379213dc9f/insights-operator/1.log"
Sep 16 14:03:44.388150 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:44.388124    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8kr9_a912322b-4b85-4e9e-857c-f18088f5e7ab/kube-rbac-proxy/0.log"
Sep 16 14:03:44.408910 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:44.408882    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8kr9_a912322b-4b85-4e9e-857c-f18088f5e7ab/exporter/0.log"
Sep 16 14:03:44.432660 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:44.432637    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8kr9_a912322b-4b85-4e9e-857c-f18088f5e7ab/extractor/0.log"
Sep 16 14:03:45.413388 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:45.413362    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hrntr/perf-node-gather-daemonset-6bmqp"
Sep 16 14:03:47.247166 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:47.247130    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694ff5d95c-p4mhm_183d736d-9ec9-4501-8d34-62c2d5bd81d5/manager/0.log"
Sep 16 14:03:47.288529 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:47.288498    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694ff5d95c-p4mhm_183d736d-9ec9-4501-8d34-62c2d5bd81d5/copy-manifests/0.log"
Sep 16 14:03:48.225568 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:48.225478    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6445dbdc5b-5z7fn_cd97e5ad-545f-4bee-b1ab-d9eca1b81626/kube-auth-proxy/0.log"
Sep 16 14:03:48.370150 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:48.370118    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff_f2aea88a-9474-43bc-a8df-d0fe082cf3a8/payload-pre-processing/0.log"
Sep 16 14:03:48.389238 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:48.389211    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49_ecb26fad-12f8-49f4-8d6e-2e52d3cc8139/payload-pre-processing/0.log"
Sep 16 14:03:48.443718 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:48.443692    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9_6e767317-491f-4b74-b6ec-64e488ea4c0a/payload-processing/0.log"
Sep 16 14:03:48.487133 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:48.487106    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-675764b9db-82s2n_01e0e7b1-ef76-4103-99cf-588da4736933/router/0.log"
Sep 16 14:03:48.820802 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:48.820747    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b97dbbc45-ntnmf_37d449d9-d828-4324-b6dd-6f71c54b7176/manager/0.log"
Sep 16 14:03:50.318252 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:50.318218    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6445dbdc5b-5z7fn_cd97e5ad-545f-4bee-b1ab-d9eca1b81626/kube-auth-proxy/0.log"
Sep 16 14:03:50.401306 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:50.401270    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-inf-b-c71ad7-649796db5d-ghsff_f2aea88a-9474-43bc-a8df-d0fe082cf3a8/payload-pre-processing/0.log"
Sep 16 14:03:50.421389 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:50.421362    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w0-a21a40-998b8fd58-h6v49_ecb26fad-12f8-49f4-8d6e-2e52d3cc8139/payload-pre-processing/0.log"
Sep 16 14:03:50.476174 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:50.476116    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-inf-b-c71ad7-545fb8f657-s7qc9_6e767317-491f-4b74-b6ec-64e488ea4c0a/payload-processing/0.log"
Sep 16 14:03:50.526555 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:50.526524    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-675764b9db-82s2n_01e0e7b1-ef76-4103-99cf-588da4736933/router/0.log"
Sep 16 14:03:54.473330 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:54.473296    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-4gjlt_d420b4ec-b559-4e74-964a-1bacec5229e5/kube-storage-version-migrator-operator/1.log"
Sep 16 14:03:54.474173 ip-10-0-143-82 kubenswrapper[2556]: I0916 14:03:54.474153    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-4gjlt_d420b4ec-b559-4e74-964a-1bacec5229e5/kube-storage-version-migrator-operator/0.log"