Sep 10 14:27:49.309200 ip-10-0-132-253 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 14:27:49.728201 ip-10-0-132-253 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:27:49.728201 ip-10-0-132-253 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 14:27:49.728201 ip-10-0-132-253 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:27:49.728201 ip-10-0-132-253 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 14:27:49.728201 ip-10-0-132-253 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:27:49.728893 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.728260    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 14:27:49.732658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732635    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:27:49.732658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732656    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732661    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732665    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732669    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732672    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732675    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732677    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732680    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732683    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732686    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732689    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732696    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732698    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732701    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732704    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732706    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732709    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732712    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732714    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732717    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:27:49.732732 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732719    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732722    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732724    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732727    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732729    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732733    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732737    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732739    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732742    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732745    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732748    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732751    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732753    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732756    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732759    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732761    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732764    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732766    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732769    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732771    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:27:49.733224 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732774    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732777    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732780    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732783    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732786    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732789    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732791    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732794    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732797    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732799    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732804    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732807    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732811    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732814    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732818    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732823    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732826    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732829    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732833    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:27:49.733704 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732836    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732840    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732843    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732846    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732848    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732863    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732866    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732869    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732871    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732874    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732877    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732880    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732883    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732885    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732888    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732892    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732895    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732897    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732900    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732903    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:27:49.734193 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732905    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732908    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732911    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732914    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732916    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732919    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732921    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732924    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732927    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732929    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732932    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732935    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732938    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732942    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732946    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732949    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732951    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732954    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.732956    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733716    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:27:49.734677 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733723    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733726    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733730    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733733    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733737    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733741    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733744    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733747    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733757    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733761    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733765    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733768    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733771    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733774    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733777    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733779    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733782    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733784    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733787    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:27:49.735185 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733790    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733793    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733795    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733798    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733801    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733804    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733807    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733809    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733812    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733815    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733818    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733820    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733823    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733825    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733828    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733831    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733833    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733836    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733838    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733841    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:27:49.735653 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733867    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733870    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733873    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733876    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733879    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733881    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733884    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733886    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733889    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733892    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733894    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733897    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733901    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733903    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733906    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733909    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733912    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733914    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733917    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733920    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:27:49.736172 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733922    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733925    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733928    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733930    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733934    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733936    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733939    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733941    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733944    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733947    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733949    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733953    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733956    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733959    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733962    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733965    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733968    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733971    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733974    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:27:49.736658 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733977    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733980    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733982    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733985    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733989    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733992    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733995    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.733998    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734001    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734003    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734005    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734008    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734011    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734013    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734016    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734018    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734021    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734023    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734025    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734028    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:27:49.737150 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734414    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734419    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734422    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734424    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734427    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734429    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734432    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734435    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734438    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734440    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734443    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734445    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734448    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734451    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734454    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734456    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734461    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734465    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734468    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:27:49.737627 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734471    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734474    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734477    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734480    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734484    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734487    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734490    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734493    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734497    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734500    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734503    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734506    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734509    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734512    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734514    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734517    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734520    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734522    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734525    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:27:49.738133 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734527    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734530    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734532    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734536    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734539    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734541    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734545    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734548    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734550    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734553    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734556    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734559    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734561    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734564    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734566    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734569    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734572    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734574    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734577    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734580    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:27:49.738588 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734582    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734585    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734587    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734590    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734593    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734595    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734598    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734600    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734602    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734605    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734607    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734610    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734613    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734615    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734618    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734620    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734624    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734632    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734634    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734637    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:27:49.739086 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734639    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734642    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734647    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734650    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734653    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734656    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734658    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734661    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734663    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734666    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734669    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734672    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734674    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734676    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734679    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734682    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734685    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734688    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734691    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734694    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:27:49.739598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.734696    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734785    2560 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734834    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734845    2560 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734850    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734866    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734870    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734874    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734879    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734883    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734887    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734890    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734894    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734897    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734902    2560 flags.go:64] FLAG: --cgroup-root=""
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734905    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734908    2560 flags.go:64] FLAG: --client-ca-file=""
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734911    2560 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734915    2560 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734919    2560 flags.go:64] FLAG: --cluster-domain=""
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734922    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734925    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734928    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734932    2560 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 14:27:49.740114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734935    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734938    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734942    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734946    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734949    2560 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734952    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734955    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734958    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734961    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734966    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734969    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734972    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734975    2560 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734978    2560 flags.go:64] FLAG: --enable-server="true"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734983    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734988    2560 flags.go:64] FLAG: --event-burst="100"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734991    2560 flags.go:64] FLAG: --event-qps="50"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734994    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.734998    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735001    2560 flags.go:64] FLAG: --eviction-hard=""
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735005    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735008    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735012    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735015    2560 flags.go:64] FLAG: --eviction-soft=""
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735018    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 14:27:49.740768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735021    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735025    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735028    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735031    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735034    2560 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735037    2560 flags.go:64] FLAG: --feature-gates=""
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735041    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735044    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735047    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735050    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735053    2560 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735056    2560 flags.go:64] FLAG: --help="false"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735059    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-132-253.ec2.internal"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735062    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735066    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735068    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735072    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735075    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735078    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735081    2560 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735084    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735088    2560 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735091    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735095    2560 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 14:27:49.741490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735098    2560 flags.go:64] FLAG: --kube-reserved=""
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735101    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735105    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735109    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735112    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735129    2560 flags.go:64] FLAG: --lock-file=""
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735133    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735136    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735139    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735181    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735186    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735190    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735194    2560 flags.go:64] FLAG: --logging-format="text"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735198    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735202    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735205    2560 flags.go:64] FLAG: --manifest-url=""
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735208    2560 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735213    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735216    2560 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735221    2560 flags.go:64] FLAG: --max-pods="110"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735224    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735228    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735231    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735234    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735237    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 14:27:49.742099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735240    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735243    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735260    2560 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735263    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735266    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735269    2560 flags.go:64] FLAG: --pod-cidr=""
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735272    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735279    2560 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735282    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735286    2560 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735289    2560 flags.go:64] FLAG: --port="10250"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735292    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735295    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035027dbdc38f2813"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735298    2560 flags.go:64] FLAG: --qos-reserved=""
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735301    2560 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735305    2560 flags.go:64] FLAG: --register-node="true"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735308    2560 flags.go:64] FLAG: --register-with-taints=""
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735313    2560 flags.go:64] FLAG: --registry-burst="10"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735316    2560 flags.go:64] FLAG: --registry-qps="5"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735320    2560 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735323    2560 flags.go:64] FLAG: --reserved-memory=""
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735326    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735330    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735333    2560 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735336    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 14:27:49.742726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735339    2560 flags.go:64] FLAG: --runonce="false"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735342    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735345    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735348    2560 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735352    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735355    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735358    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735361    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735364    2560 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735367    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735370    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735373    2560 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735376    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735379    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735383    2560 flags.go:64] FLAG: --system-cgroups=""
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735386    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735394    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735397    2560 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735400    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735404    2560 flags.go:64] FLAG: --tls-min-version=""
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735408    2560 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735411    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735414    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735417    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735421    2560 flags.go:64] FLAG: --v="2"
Sep 10 14:27:49.743657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735426    2560 flags.go:64] FLAG: --version="false"
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735431    2560 flags.go:64] FLAG: --vmodule=""
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735435    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.735438    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735604    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735608    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735611    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735614    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735617    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735620    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735622    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735625    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735628    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735630    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735633    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735635    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735638    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735640    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735643    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735645    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:27:49.744303 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735648    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735651    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735653    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735656    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735659    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735661    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735664    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735667    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735670    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735672    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735675    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735680    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735682    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735691    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735702    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735705    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735708    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735711    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735714    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:27:49.745066 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735718    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735722    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735725    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735728    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735730    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735733    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735735    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735738    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735741    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735743    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735746    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735748    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735752    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735756    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735759    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735762    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735764    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735767    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735769    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:27:49.745887 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735772    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735775    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735778    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735781    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735784    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735788    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735791    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735793    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735796    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735799    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735802    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735805    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735808    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735810    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735813    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735815    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735818    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735820    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735823    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735825    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:27:49.746666 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735828    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735831    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735834    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735836    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735839    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735842    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735844    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735847    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735849    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735866    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735870    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735873    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735876    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735879    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735883    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735887    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735890    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735894    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735897    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735900    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:27:49.747556 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735903    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735908    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735911    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735913    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.735916    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.736618    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.746567    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.746588    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.746616    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.746628    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746737    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746746    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746753    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746758    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:27:49.748393 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746763    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746768    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746772    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746776    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746781    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746785    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746789    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746794    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746799    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746803    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746807    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746812    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746816    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746820    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746824    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746829    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746833    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746838    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746844    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746877    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:27:49.748761 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746884    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746889    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746893    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746900    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746907    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746912    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746917    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746922    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746926    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746930    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746935    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746940    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746944    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746948    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746952    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746957    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746961    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746965    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746969    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:27:49.749514 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746974    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746978    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746982    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746986    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746991    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746995    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.746999    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747003    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747007    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747011    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747015    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747019    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747023    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747035    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747039    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747043    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747047    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747052    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747057    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747069    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:27:49.750339 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747074    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747078    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747083    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747087    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747091    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747095    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747099    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747103    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747107    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747111    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747115    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747119    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747123    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747128    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747132    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747136    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747140    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747144    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747151    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:27:49.750964 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747157    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747161    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747165    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747169    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747173    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747177    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747182    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747193    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747197    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747201    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747206    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747211    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747215    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747219    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747224    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747228    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:27:49.751424 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747232    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.747241    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747469    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747477    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747482    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747486    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747490    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747495    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747500    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747504    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747508    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747512    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747516    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747521    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747525    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:27:49.751889 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747529    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747533    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747537    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747541    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747545    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747549    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747553    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747558    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747561    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747573    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747577    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747581    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747585    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747590    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747596    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747600    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747605    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747609    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747615    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747621    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:27:49.752266 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747626    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747631    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747635    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747640    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747645    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747649    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747654    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747660    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747665    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747669    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747673    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747679    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747684    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747689    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747702    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747707    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747712    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747716    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747720    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:27:49.752820 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747724    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747729    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747733    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747745    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747749    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747754    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747758    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747762    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747766    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747771    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747775    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747779    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747783    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747787    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747791    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747795    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747799    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747803    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747808    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747812    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:27:49.753472 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747816    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747820    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747824    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747829    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747833    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747837    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747841    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747845    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747849    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747869    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747873    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747877    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747881    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747885    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747889    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747894    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747906    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747911    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747916    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747919    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:27:49.753991 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747923    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747927    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747932    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747936    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747940    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747944    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:49.747948    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.747957    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.748178    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.753211    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.754352    2560 server.go:1019] "Starting client certificate rotation"
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.754472    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:27:49.754532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.754526    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:27:49.757434 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.757414    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:27:49.764658 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.764632    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:27:49.777556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.777535    2560 log.go:25] "Validated CRI v1 runtime API"
Sep 10 14:27:49.781015 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.780998    2560 log.go:25] "Validated CRI v1 image API"
Sep 10 14:27:49.781902 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.781885    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 14:27:49.785974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.785955    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:27:49.788327 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.788301    2560 fs.go:135] Filesystem UUIDs: map[0680b89f-df39-45b3-afba-a99e4fdd0e73:/dev/nvme0n1p3 0fd7257f-5dce-49dd-af16-faa38fb598cd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 10 14:27:49.788374 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.788327    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 14:27:49.793464 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.793350    2560 manager.go:216] Machine: {Timestamp:2026-09-10 14:27:49.792047823 +0000 UTC m=+0.375285397 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3120377 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20081fbd0143de24972b5b5729f1dd SystemUUID:ec20081f-bd01-43de-2497-2b5b5729f1dd BootID:e71220ea-9223-4362-ba28-ad6c13e18d50 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:75:b9:5b:b4:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:75:b9:5b:b4:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:98:3b:1b:38:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 14:27:49.793464 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.793459    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 14:27:49.793571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.793545    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 14:27:49.797090 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.797060    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 14:27:49.797251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.797094    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-253.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 14:27:49.797973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.797961    2560 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 14:27:49.798018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.797975    2560 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 14:27:49.798018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.797983    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:27:49.798892 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.798881    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:27:49.800878 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.800848    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 14:27:49.806599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806580    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:27:49.806742 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806731    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 14:27:49.806789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806763    2560 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 14:27:49.806789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806774    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 14:27:49.806789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806788    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 14:27:49.806927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806798    2560 kubelet.go:398] "Adding apiserver pod source"
Sep 10 14:27:49.806927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.806810    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 14:27:49.807301 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.807275    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:27:49.807301 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.807305    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:27:49.810626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.810601    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 14:27:49.817185 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:49.817157    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-253.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 14:27:49.817277 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:49.817222    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 14:27:49.835453 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.835350    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kdcs"
Sep 10 14:27:49.840360 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.840336    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kdcs"
Sep 10 14:27:49.957937 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.957914    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 14:27:49.958049 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.957950    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 14:27:49.958792 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958779    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 14:27:49.958829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958798    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 14:27:49.958829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958805    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 14:27:49.958829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958810    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 14:27:49.958829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958819    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 14:27:49.958829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958824    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 14:27:49.958974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958839    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 14:27:49.958974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958845    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 14:27:49.958974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958866    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 14:27:49.958974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958872    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 14:27:49.958974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958880    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 14:27:49.958974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.958886    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 14:27:49.959752 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.959741    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 14:27:49.959786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.959753    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 14:27:49.963608 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.963589    2560 server.go:1268] "Started kubelet"
Sep 10 14:27:49.963752 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.963699    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 14:27:49.963893 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.963704    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 14:27:49.964019 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.963915    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 14:27:49.964464 ip-10-0-132-253 systemd[1]: Started Kubernetes Kubelet.
Sep 10 14:27:49.965893 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.965719    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 14:27:49.965977 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.965950    2560 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 14:27:49.970529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.970511    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 14:27:49.971147 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:49.971125    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:49.971147 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:49.970835    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 14:27:49.971291 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.970515    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 14:27:49.971954 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.971740    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 14:27:49.972023 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.971958    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 14:27:49.972023 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.971747    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 14:27:49.972124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.972117    2560 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 14:27:49.972161 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.972128    2560 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 14:27:49.973727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.973704    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:49.973727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.973724    2560 factory.go:55] Registering systemd factory
Sep 10 14:27:49.973920 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.973755    2560 factory.go:223] Registration of the systemd container factory successfully
Sep 10 14:27:49.974040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.974020    2560 factory.go:153] Registering CRI-O factory
Sep 10 14:27:49.974040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.974034    2560 factory.go:223] Registration of the crio container factory successfully
Sep 10 14:27:49.974128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.974085    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 14:27:49.974128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.974099    2560 factory.go:103] Registering Raw factory
Sep 10 14:27:49.974128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.974112    2560 manager.go:1196] Started watching for new ooms in manager
Sep 10 14:27:49.974544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.974533    2560 manager.go:349] Starting recovery of all containers
Sep 10 14:27:49.976192 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.976175    2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-132-253.ec2.internal" not found
Sep 10 14:27:49.976470 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:49.976454    2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-253.ec2.internal\" not found" node="ip-10-0-132-253.ec2.internal"
Sep 10 14:27:49.985948 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.985926    2560 manager.go:354] Recovery completed
Sep 10 14:27:49.990916 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.990899    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:49.992240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.992225    2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-132-253.ec2.internal" not found
Sep 10 14:27:49.992820 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.992802    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:49.992922 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.992832    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:49.992922 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.992848    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:49.993537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.993519    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 14:27:49.993537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.993529    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:49.993537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.993535    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 14:27:49.993735 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.993557    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:27:49.994264 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.994251    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:49.994321 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.994277    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:49.994321 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.994289    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:49.995795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.995782    2560 policy_none.go:49] "None policy: Start"
Sep 10 14:27:49.995849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.995800    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 14:27:49.995849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.995813    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 14:27:49.997903 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:49.997888    2560 policy_none.go:47] "Start"
Sep 10 14:27:50.037498 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.037474    2560 manager.go:337] "Starting Device Plugin manager"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.037520    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.037534    2560 server.go:87] "Starting device plugin registration server"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.037887    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.037900    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.037998    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.038076    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.038084    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.038616    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 14:27:50.050415 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.038674    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:50.052284 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.052266    2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-132-253.ec2.internal" not found
Sep 10 14:27:50.083110 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.083078    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 14:27:50.084234 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.084216    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 14:27:50.084288 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.084279    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 14:27:50.084330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.084310    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 14:27:50.084438 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.084403    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 14:27:50.086771 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.086748    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:50.138919 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.138891    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:50.139925 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.139909    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:50.140034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.139943    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:50.140034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.139957    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:50.140034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.139987    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.147805 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.147781    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.147805 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.147810    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-253.ec2.internal\": node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:50.176501 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.176472    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:50.185539 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.185493    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"]
Sep 10 14:27:50.185657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.185585    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:50.186667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.186651    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:50.186732 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.186680    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:50.186732 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.186692    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:50.189047 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.189034    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:50.189181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.189167    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.189218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.189197    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:50.190016 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.189986    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:50.190016 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.189995    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:50.190180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.190034    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:50.190180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.190045    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:50.190180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.190013    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:50.190180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.190092    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:50.192432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.192415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.192523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.192446    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:27:50.193124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.193106    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:27:50.193192 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.193131    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:27:50.193192 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.193145    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:27:50.210515 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.210489    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-253.ec2.internal\" not found" node="ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.214455 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.214437    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-253.ec2.internal\" not found" node="ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.273556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.273472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c36c54bc5a50414caa33ecd6875e0504-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal\" (UID: \"c36c54bc5a50414caa33ecd6875e0504\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.273556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.273515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c36c54bc5a50414caa33ecd6875e0504-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal\" (UID: \"c36c54bc5a50414caa33ecd6875e0504\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.273556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.273535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/424c21ac2ec6f354644b2292aeae7f24-config\") pod \"kube-apiserver-proxy-ip-10-0-132-253.ec2.internal\" (UID: \"424c21ac2ec6f354644b2292aeae7f24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.276580 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.276562    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:50.373964 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.373932    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c36c54bc5a50414caa33ecd6875e0504-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal\" (UID: \"c36c54bc5a50414caa33ecd6875e0504\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.374083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.373967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c36c54bc5a50414caa33ecd6875e0504-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal\" (UID: \"c36c54bc5a50414caa33ecd6875e0504\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.374083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.373989    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/424c21ac2ec6f354644b2292aeae7f24-config\") pod \"kube-apiserver-proxy-ip-10-0-132-253.ec2.internal\" (UID: \"424c21ac2ec6f354644b2292aeae7f24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.374083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.374044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c36c54bc5a50414caa33ecd6875e0504-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal\" (UID: \"c36c54bc5a50414caa33ecd6875e0504\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.374200 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.374091    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/424c21ac2ec6f354644b2292aeae7f24-config\") pod \"kube-apiserver-proxy-ip-10-0-132-253.ec2.internal\" (UID: \"424c21ac2ec6f354644b2292aeae7f24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.374200 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:50.374118    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c36c54bc5a50414caa33ecd6875e0504-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal\" (UID: \"c36c54bc5a50414caa33ecd6875e0504\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:50.377008 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:50.376979    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:51.704953 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:51.704918    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:51.741102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.741062    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:51.744734 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.744711    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:51.806054 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:51.805974    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-253.ec2.internal\" not found"
Sep 10 14:27:51.866871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.866848    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:51.899310 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.899264    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:51.912402 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.912379    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:27:51.913969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.913956    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal"
Sep 10 14:27:51.924747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.924726    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:27:51.981944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.981913    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 14:27:51.982404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.982075    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:27:51.982404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.982083    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:27:51.982404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:51.982078    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:27:52.034696 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.034645    2560 apiserver.go:52] "Watching apiserver"
Sep 10 14:27:52.047445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.047420    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 14:27:52.047768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.047747    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-6bbmm","kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw","openshift-cluster-node-tuning-operator/tuned-lzvz4","openshift-dns/node-resolver-lwfht","openshift-multus/multus-additional-cni-plugins-6bhrh","openshift-multus/multus-d8sfz","openshift-image-registry/node-ca-pksmv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal","openshift-multus/network-metrics-daemon-57lnd","openshift-network-diagnostics/network-check-target-c6v7j","openshift-network-operator/iptables-alerter-x5vpk","openshift-ovn-kubernetes/ovnkube-node-f8xhq","kube-system/global-pull-secret-syncer-ndntw"]
Sep 10 14:27:52.050228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.050209    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.052472 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.052448    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.052886 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.052873    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 14:27:52.058766 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.058695    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rph4b\""
Sep 10 14:27:52.059629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.059039    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 14:27:52.059629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.059337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 14:27:52.059629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.059404    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.059629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.059504    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.059629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.059555    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j5sdw\""
Sep 10 14:27:52.059909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.059826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.061811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.061791    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.061811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.061804    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bzc9v\""
Sep 10 14:27:52.061940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.061892    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.062108 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.062093    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.063891 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.063872    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.063989 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.063939    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-njw85\""
Sep 10 14:27:52.063989 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.063944    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.064386 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.064366    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.066293 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066266    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 14:27:52.066365 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066266    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 14:27:52.066415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066398    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.066415 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066406    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zp2m4\""
Sep 10 14:27:52.066627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.066733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066690    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 14:27:52.066733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.066713    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.068391 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.068372    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 14:27:52.068484 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.068409    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4qd9v\""
Sep 10 14:27:52.068927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.068915    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.070708 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.070688    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-75k5q\""
Sep 10 14:27:52.070806 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.070688    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 14:27:52.070806 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.070693    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.070942 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.070826    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.071169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.071154    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.071229 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.071213    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:27:52.073488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.073473    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:52.073559 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.073523    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:27:52.073559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.073459    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 14:22:51 +0000 UTC" deadline="2028-03-24 20:01:51.561018748 +0000 UTC"
Sep 10 14:27:52.073559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.073547    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13469h33m59.487474628s"
Sep 10 14:27:52.075666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.075652    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.078005 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.077986    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.079236 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.079220    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dqdvj\""
Sep 10 14:27:52.079446 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.079429    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.079670 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.079657    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.079761 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.079746    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 14:27:52.080203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.080191    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 14:27:52.080265 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.080249    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.080349 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.080328    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:27:52.080554 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.080540    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 14:27:52.081901 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.081887    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 14:27:52.082370 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.082354    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6lsg4\""
Sep 10 14:27:52.082452 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.082402    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 14:27:52.082452 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.082403    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 14:27:52.082544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.082411    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 14:27:52.100766 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.100742    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 14:27:52.103921 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.103901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.104004 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.103945    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-slash\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104004 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.103987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-run-netns\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104035    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-ovnkube-script-lib\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-system-cni-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.104102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-netns\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.104189 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76fk\" (UniqueName: \"kubernetes.io/projected/6cddba93-d2fc-4e91-be06-a8559e276d53-kube-api-access-p76fk\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.104189 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e289bd29-96ce-4777-b205-7a700842368d-ovn-node-metrics-cert\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104297 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8d7361c0-0247-4988-bd32-fda7907cb325-hosts-file\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.104297 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.104297 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w8zc\" (UniqueName: \"kubernetes.io/projected/53311511-e278-4666-bb1f-30c6c101fede-kube-api-access-6w8zc\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-ovn\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysctl-d\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysctl-conf\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104360    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-cni-multus\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104374    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-multus-certs\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cddba93-d2fc-4e91-be06-a8559e276d53-host\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.104417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104403    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-node-log\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104435    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-modprobe-d\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-tuned\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104507    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ec070a-311b-45f2-8375-8f94b0d9a69b-tmp\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104538    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl6h4\" (UniqueName: \"kubernetes.io/projected/d0a7d218-c488-478d-ad91-3e119f318ad0-kube-api-access-wl6h4\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-k8s-cni-cncf-io\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-run-ovn-kubernetes\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-cni-netd\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysconfig\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-system-cni-dir\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.104679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-socket-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8g4x\" (UniqueName: \"kubernetes.io/projected/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-kube-api-access-j8g4x\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-daemon-config\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-log-socket\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-env-overrides\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-iptables-alerter-script\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2db006d1-41d7-487e-a3ee-ded18e3db48a-cni-binary-copy\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-conf-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104900    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-etc-kubernetes\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104927    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-kubernetes\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flp42\" (UniqueName: \"kubernetes.io/projected/d9ec070a-311b-45f2-8375-8f94b0d9a69b-kube-api-access-flp42\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.104985    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-var-lib-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-run\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105112    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-systemd\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-registration-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-device-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-cni-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-kubelet\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105409    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-host-slash\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105447    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdwz\" (UniqueName: \"kubernetes.io/projected/e289bd29-96ce-4777-b205-7a700842368d-kube-api-access-2sdwz\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105479    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6cddba93-d2fc-4e91-be06-a8559e276d53-serviceca\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-hostroot\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105525    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105551    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/807de0e5-36be-4155-b273-fc7eb43f2f53-kubelet-config\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105600    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-cni-bin\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.105625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7361c0-0247-4988-bd32-fda7907cb325-tmp-dir\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105643    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105663    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-cnibin\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-systemd\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-etc-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105735    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105773    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2e559fb-9b78-4509-b544-3410d402bda8-agent-certs\") pod \"konnectivity-agent-6bbmm\" (UID: \"a2e559fb-9b78-4509-b544-3410d402bda8\") " pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105824    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smzrj\" (UniqueName: \"kubernetes.io/projected/2db006d1-41d7-487e-a3ee-ded18e3db48a-kube-api-access-smzrj\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105851    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-systemd-units\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-sys\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-var-lib-kubelet\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105942    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-host\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105976    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.105992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106013    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-kubelet\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.106093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106031    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-cni-bin\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-cnibin\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106072    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hksg7\" (UniqueName: \"kubernetes.io/projected/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-kube-api-access-hksg7\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106097    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/807de0e5-36be-4155-b273-fc7eb43f2f53-dbus\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2e559fb-9b78-4509-b544-3410d402bda8-konnectivity-ca\") pod \"konnectivity-agent-6bbmm\" (UID: \"a2e559fb-9b78-4509-b544-3410d402bda8\") " pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-sys-fs\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106159    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-socket-dir-parent\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-ovnkube-config\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106201    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-lib-modules\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106216    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5ggt\" (UniqueName: \"kubernetes.io/projected/8d7361c0-0247-4988-bd32-fda7907cb325-kube-api-access-r5ggt\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.106589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.106249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.199153 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.199132    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 14:27:52.206978 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.206951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5ggt\" (UniqueName: \"kubernetes.io/projected/8d7361c0-0247-4988-bd32-fda7907cb325-kube-api-access-r5ggt\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.207131 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.206982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.207131 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.207131 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-slash\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207131 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-run-netns\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207131 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-ovnkube-script-lib\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207324    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-system-cni-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207384 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-netns\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.207429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76fk\" (UniqueName: \"kubernetes.io/projected/6cddba93-d2fc-4e91-be06-a8559e276d53-kube-api-access-p76fk\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.207429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-slash\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-run-netns\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e289bd29-96ce-4777-b205-7a700842368d-ovn-node-metrics-cert\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-netns\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207459    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8d7361c0-0247-4988-bd32-fda7907cb325-hosts-file\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-system-cni-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207485    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207513    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w8zc\" (UniqueName: \"kubernetes.io/projected/53311511-e278-4666-bb1f-30c6c101fede-kube-api-access-6w8zc\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-ovn\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8d7361c0-0247-4988-bd32-fda7907cb325-hosts-file\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.207561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysctl-d\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysctl-conf\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-cni-multus\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-ovn\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-multus-certs\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cddba93-d2fc-4e91-be06-a8559e276d53-host\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-node-log\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207698    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-modprobe-d\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207721    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-tuned\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207738    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysctl-conf\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207740    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ec070a-311b-45f2-8375-8f94b0d9a69b-tmp\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207775    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl6h4\" (UniqueName: \"kubernetes.io/projected/d0a7d218-c488-478d-ad91-3e119f318ad0-kube-api-access-wl6h4\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-ovnkube-script-lib\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-cni-multus\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cddba93-d2fc-4e91-be06-a8559e276d53-host\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-k8s-cni-cncf-io\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207776    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-multus-certs\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.207857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207855    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-node-log\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-run-k8s-cni-cncf-io\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysctl-d\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207870    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-run-ovn-kubernetes\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-cni-netd\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysconfig\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207973    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-system-cni-dir\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.207996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-socket-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8g4x\" (UniqueName: \"kubernetes.io/projected/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-kube-api-access-j8g4x\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-modprobe-d\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-daemon-config\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-log-socket\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-cni-netd\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208095    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-env-overrides\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-iptables-alerter-script\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208148    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2db006d1-41d7-487e-a3ee-ded18e3db48a-cni-binary-copy\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.208586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208174    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-conf-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208176    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208196    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-etc-kubernetes\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208237    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-etc-kubernetes\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-log-socket\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-kubernetes\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208309    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-sysconfig\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flp42\" (UniqueName: \"kubernetes.io/projected/d9ec070a-311b-45f2-8375-8f94b0d9a69b-kube-api-access-flp42\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-socket-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208386    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-var-lib-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-env-overrides\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-var-lib-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208524    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-run\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208574    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-systemd\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-registration-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-device-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.209399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208654    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-cni-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-kubelet\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-host-slash\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdwz\" (UniqueName: \"kubernetes.io/projected/e289bd29-96ce-4777-b205-7a700842368d-kube-api-access-2sdwz\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208759    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6cddba93-d2fc-4e91-be06-a8559e276d53-serviceca\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-hostroot\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208809    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208825    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2db006d1-41d7-487e-a3ee-ded18e3db48a-cni-binary-copy\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208858    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208873    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-conf-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208909    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/807de0e5-36be-4155-b273-fc7eb43f2f53-kubelet-config\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-kubelet\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-host-slash\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208961    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-run\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.208864    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/807de0e5-36be-4155-b273-fc7eb43f2f53-kubelet-config\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-cni-bin\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.209033    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:52.210203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209042    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7361c0-0247-4988-bd32-fda7907cb325-tmp-dir\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209068    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.209099    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:52.709080844 +0000 UTC m=+2.064893746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209117    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-cnibin\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-kubernetes\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.209146    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209178    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-hostroot\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209182    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.209192    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:52.709177277 +0000 UTC m=+2.064990182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209201    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-cni-dir\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-cni-bin\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209251    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-device-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-systemd\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-registration-dir\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-systemd\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209391    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-etc-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-run-ovn-kubernetes\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7361c0-0247-4988-bd32-fda7907cb325-tmp-dir\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209507    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2e559fb-9b78-4509-b544-3410d402bda8-agent-certs\") pod \"konnectivity-agent-6bbmm\" (UID: \"a2e559fb-9b78-4509-b544-3410d402bda8\") " pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209588    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-cnibin\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209594    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-daemon-config\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209613    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-run-systemd\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209655    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-etc-openvswitch\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smzrj\" (UniqueName: \"kubernetes.io/projected/2db006d1-41d7-487e-a3ee-ded18e3db48a-kube-api-access-smzrj\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-systemd-units\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209895    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209908    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-sys\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-var-lib-kubelet\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-sys\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6cddba93-d2fc-4e91-be06-a8559e276d53-serviceca\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210061    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-var-lib-kubelet\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.211776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210070    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-systemd-units\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210104    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-host\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210131    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.209980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-system-cni-dir\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210201    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-host\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210204    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-kubelet\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210240    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-cni-bin\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210268    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-host-var-lib-kubelet\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-cnibin\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210315    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e289bd29-96ce-4777-b205-7a700842368d-host-cni-bin\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hksg7\" (UniqueName: \"kubernetes.io/projected/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-kube-api-access-hksg7\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210337    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53311511-e278-4666-bb1f-30c6c101fede-cnibin\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210354    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/807de0e5-36be-4155-b273-fc7eb43f2f53-dbus\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210875    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2e559fb-9b78-4509-b544-3410d402bda8-konnectivity-ca\") pod \"konnectivity-agent-6bbmm\" (UID: \"a2e559fb-9b78-4509-b544-3410d402bda8\") " pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210913    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-sys-fs\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.212393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-socket-dir-parent\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-ovnkube-config\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.210992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-lib-modules\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211071    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-sys-fs\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211086    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-iptables-alerter-script\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/807de0e5-36be-4155-b273-fc7eb43f2f53-dbus\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53311511-e278-4666-bb1f-30c6c101fede-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211197    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2db006d1-41d7-487e-a3ee-ded18e3db48a-multus-socket-dir-parent\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9ec070a-311b-45f2-8375-8f94b0d9a69b-lib-modules\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211735    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e289bd29-96ce-4777-b205-7a700842368d-ovnkube-config\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.211906    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e289bd29-96ce-4777-b205-7a700842368d-ovn-node-metrics-cert\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.212319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ec070a-311b-45f2-8375-8f94b0d9a69b-tmp\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.212468    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2e559fb-9b78-4509-b544-3410d402bda8-konnectivity-ca\") pod \"konnectivity-agent-6bbmm\" (UID: \"a2e559fb-9b78-4509-b544-3410d402bda8\") " pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.212622    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2e559fb-9b78-4509-b544-3410d402bda8-agent-certs\") pod \"konnectivity-agent-6bbmm\" (UID: \"a2e559fb-9b78-4509-b544-3410d402bda8\") " pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.212871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.212626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d9ec070a-311b-45f2-8375-8f94b0d9a69b-etc-tuned\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.214686 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.214669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5ggt\" (UniqueName: \"kubernetes.io/projected/8d7361c0-0247-4988-bd32-fda7907cb325-kube-api-access-r5ggt\") pod \"node-resolver-lwfht\" (UID: \"8d7361c0-0247-4988-bd32-fda7907cb325\") " pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.218176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.218149    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w8zc\" (UniqueName: \"kubernetes.io/projected/53311511-e278-4666-bb1f-30c6c101fede-kube-api-access-6w8zc\") pod \"multus-additional-cni-plugins-6bhrh\" (UID: \"53311511-e278-4666-bb1f-30c6c101fede\") " pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.218584 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.218561    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76fk\" (UniqueName: \"kubernetes.io/projected/6cddba93-d2fc-4e91-be06-a8559e276d53-kube-api-access-p76fk\") pod \"node-ca-pksmv\" (UID: \"6cddba93-d2fc-4e91-be06-a8559e276d53\") " pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.220162 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.220134    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdwz\" (UniqueName: \"kubernetes.io/projected/e289bd29-96ce-4777-b205-7a700842368d-kube-api-access-2sdwz\") pod \"ovnkube-node-f8xhq\" (UID: \"e289bd29-96ce-4777-b205-7a700842368d\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.220414 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.220396    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl6h4\" (UniqueName: \"kubernetes.io/projected/d0a7d218-c488-478d-ad91-3e119f318ad0-kube-api-access-wl6h4\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.223227 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.223210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flp42\" (UniqueName: \"kubernetes.io/projected/d9ec070a-311b-45f2-8375-8f94b0d9a69b-kube-api-access-flp42\") pod \"tuned-lzvz4\" (UID: \"d9ec070a-311b-45f2-8375-8f94b0d9a69b\") " pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.223634 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.223621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8g4x\" (UniqueName: \"kubernetes.io/projected/560dd0fe-d588-4a95-9882-4b8a0c64f1c2-kube-api-access-j8g4x\") pod \"aws-ebs-csi-driver-node-k26fw\" (UID: \"560dd0fe-d588-4a95-9882-4b8a0c64f1c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.223796 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.223779    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:27:52.223844 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.223804    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:27:52.223844 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.223817    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:52.223924 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.223895    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:52.723877342 +0000 UTC m=+2.079690246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:52.225803 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.225783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hksg7\" (UniqueName: \"kubernetes.io/projected/d25fd70d-c6c7-46fb-80bd-43227ad3c20c-kube-api-access-hksg7\") pod \"iptables-alerter-x5vpk\" (UID: \"d25fd70d-c6c7-46fb-80bd-43227ad3c20c\") " pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.226089 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.226072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smzrj\" (UniqueName: \"kubernetes.io/projected/2db006d1-41d7-487e-a3ee-ded18e3db48a-kube-api-access-smzrj\") pod \"multus-d8sfz\" (UID: \"2db006d1-41d7-487e-a3ee-ded18e3db48a\") " pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.234495 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.234479    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:27:52.236897 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.236878    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5d4mc"
Sep 10 14:27:52.244554 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.244532    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5d4mc"
Sep 10 14:27:52.272111 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.272084    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:52.374943 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.374865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:27:52.402512 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.402479    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw"
Sep 10 14:27:52.424404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.424379    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lzvz4"
Sep 10 14:27:52.429319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.429291    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwfht"
Sep 10 14:27:52.431751 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.431711    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36c54bc5a50414caa33ecd6875e0504.slice/crio-509daa38a3b11d766fc9bf7db5a28318f0f0e57fc4910e3c67193bc60ccd02b6 WatchSource:0}: Error finding container 509daa38a3b11d766fc9bf7db5a28318f0f0e57fc4910e3c67193bc60ccd02b6: Status 404 returned error can't find the container with id 509daa38a3b11d766fc9bf7db5a28318f0f0e57fc4910e3c67193bc60ccd02b6
Sep 10 14:27:52.432210 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.432139    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560dd0fe_d588_4a95_9882_4b8a0c64f1c2.slice/crio-4f69b048ce88295cfe69caa37d7256074a9035bae67e6278d5c61846dba4ebd4 WatchSource:0}: Error finding container 4f69b048ce88295cfe69caa37d7256074a9035bae67e6278d5c61846dba4ebd4: Status 404 returned error can't find the container with id 4f69b048ce88295cfe69caa37d7256074a9035bae67e6278d5c61846dba4ebd4
Sep 10 14:27:52.433164 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.433142    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424c21ac2ec6f354644b2292aeae7f24.slice/crio-1d703351e7cc69486e0b83ed9fc6a9cc814130edd25fb1f9d0798801e8d83e24 WatchSource:0}: Error finding container 1d703351e7cc69486e0b83ed9fc6a9cc814130edd25fb1f9d0798801e8d83e24: Status 404 returned error can't find the container with id 1d703351e7cc69486e0b83ed9fc6a9cc814130edd25fb1f9d0798801e8d83e24
Sep 10 14:27:52.437002 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.436980    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ec070a_311b_45f2_8375_8f94b0d9a69b.slice/crio-e8257c3f4c94d03895de826ffdb3943ddaaa5b6ccd6f49069dde9722695c1e57 WatchSource:0}: Error finding container e8257c3f4c94d03895de826ffdb3943ddaaa5b6ccd6f49069dde9722695c1e57: Status 404 returned error can't find the container with id e8257c3f4c94d03895de826ffdb3943ddaaa5b6ccd6f49069dde9722695c1e57
Sep 10 14:27:52.438213 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.438197    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:27:52.440529 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.440509    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7361c0_0247_4988_bd32_fda7907cb325.slice/crio-091689305d0e26df7ea1b3bd8fbdfbe4faa3a8e89a11bb65a28e0b41eeff4966 WatchSource:0}: Error finding container 091689305d0e26df7ea1b3bd8fbdfbe4faa3a8e89a11bb65a28e0b41eeff4966: Status 404 returned error can't find the container with id 091689305d0e26df7ea1b3bd8fbdfbe4faa3a8e89a11bb65a28e0b41eeff4966
Sep 10 14:27:52.457050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.457027    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6bhrh"
Sep 10 14:27:52.466739 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.466591    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53311511_e278_4666_bb1f_30c6c101fede.slice/crio-63bca88630ea695bff64025e09ffb7af0b920cfde7210cf376d8e7f4bed0a172 WatchSource:0}: Error finding container 63bca88630ea695bff64025e09ffb7af0b920cfde7210cf376d8e7f4bed0a172: Status 404 returned error can't find the container with id 63bca88630ea695bff64025e09ffb7af0b920cfde7210cf376d8e7f4bed0a172
Sep 10 14:27:52.474626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.474607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d8sfz"
Sep 10 14:27:52.482334 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.482312    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db006d1_41d7_487e_a3ee_ded18e3db48a.slice/crio-07a6a289fe4713ac2a99ee51e619f15161b5709ec1d598889d9da4df74a27167 WatchSource:0}: Error finding container 07a6a289fe4713ac2a99ee51e619f15161b5709ec1d598889d9da4df74a27167: Status 404 returned error can't find the container with id 07a6a289fe4713ac2a99ee51e619f15161b5709ec1d598889d9da4df74a27167
Sep 10 14:27:52.490043 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.490026    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pksmv"
Sep 10 14:27:52.497565 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.497545    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cddba93_d2fc_4e91_be06_a8559e276d53.slice/crio-926e8f31445918c7fae7093435d77bfb452b141beb290339093a525825eab27d WatchSource:0}: Error finding container 926e8f31445918c7fae7093435d77bfb452b141beb290339093a525825eab27d: Status 404 returned error can't find the container with id 926e8f31445918c7fae7093435d77bfb452b141beb290339093a525825eab27d
Sep 10 14:27:52.514191 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.514170    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x5vpk"
Sep 10 14:27:52.522590 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.522569    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25fd70d_c6c7_46fb_80bd_43227ad3c20c.slice/crio-d621ade1eb9394613da7ea685ff456798a417d1bc4b37de93cdddcb74f79258b WatchSource:0}: Error finding container d621ade1eb9394613da7ea685ff456798a417d1bc4b37de93cdddcb74f79258b: Status 404 returned error can't find the container with id d621ade1eb9394613da7ea685ff456798a417d1bc4b37de93cdddcb74f79258b
Sep 10 14:27:52.554123 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.554097    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode289bd29_96ce_4777_b205_7a700842368d.slice/crio-349aec116818a749098bf0b5c6f16c1550a96c6987dee158c8f508027622cda7 WatchSource:0}: Error finding container 349aec116818a749098bf0b5c6f16c1550a96c6987dee158c8f508027622cda7: Status 404 returned error can't find the container with id 349aec116818a749098bf0b5c6f16c1550a96c6987dee158c8f508027622cda7
Sep 10 14:27:52.714372 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.714343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:52.714556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.714386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:52.714556 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.714480    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:52.714556 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.714519    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:52.714556 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.714548    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:53.714531141 +0000 UTC m=+3.070344032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:52.714713 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.714572    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:53.714554664 +0000 UTC m=+3.070367548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:52.815208 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:52.815170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:52.815410 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.815350    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:27:52.815410 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.815368    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:27:52.815410 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.815381    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:52.815568 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:52.815444    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:53.815426741 +0000 UTC m=+3.171239627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:52.973192 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:27:52.973155    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e559fb_9b78_4509_b544_3410d402bda8.slice/crio-4a8f14f16c49e5306c7812ca12b268e1a2ae819e6a15ce3fbc37fa03fc020b16 WatchSource:0}: Error finding container 4a8f14f16c49e5306c7812ca12b268e1a2ae819e6a15ce3fbc37fa03fc020b16: Status 404 returned error can't find the container with id 4a8f14f16c49e5306c7812ca12b268e1a2ae819e6a15ce3fbc37fa03fc020b16
Sep 10 14:27:53.010957 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.010731    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:53.247455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.247361    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:22:52 +0000 UTC" deadline="2028-03-16 20:45:13.535042145 +0000 UTC"
Sep 10 14:27:53.247455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.247403    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13278h17m20.287644352s"
Sep 10 14:27:53.313539 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.313508    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:53.313730 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.313650    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:27:53.314849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.314673    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:53.331245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.331155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d8sfz" event={"ID":"2db006d1-41d7-487e-a3ee-ded18e3db48a","Type":"ContainerStarted","Data":"07a6a289fe4713ac2a99ee51e619f15161b5709ec1d598889d9da4df74a27167"}
Sep 10 14:27:53.352926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.352888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lzvz4" event={"ID":"d9ec070a-311b-45f2-8375-8f94b0d9a69b","Type":"ContainerStarted","Data":"e8257c3f4c94d03895de826ffdb3943ddaaa5b6ccd6f49069dde9722695c1e57"}
Sep 10 14:27:53.366422 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.366387    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bbmm" event={"ID":"a2e559fb-9b78-4509-b544-3410d402bda8","Type":"ContainerStarted","Data":"4a8f14f16c49e5306c7812ca12b268e1a2ae819e6a15ce3fbc37fa03fc020b16"}
Sep 10 14:27:53.391370 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.391324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x5vpk" event={"ID":"d25fd70d-c6c7-46fb-80bd-43227ad3c20c","Type":"ContainerStarted","Data":"d621ade1eb9394613da7ea685ff456798a417d1bc4b37de93cdddcb74f79258b"}
Sep 10 14:27:53.394998 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.394907    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pksmv" event={"ID":"6cddba93-d2fc-4e91-be06-a8559e276d53","Type":"ContainerStarted","Data":"926e8f31445918c7fae7093435d77bfb452b141beb290339093a525825eab27d"}
Sep 10 14:27:53.403593 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.403559    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerStarted","Data":"63bca88630ea695bff64025e09ffb7af0b920cfde7210cf376d8e7f4bed0a172"}
Sep 10 14:27:53.411069 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.411036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwfht" event={"ID":"8d7361c0-0247-4988-bd32-fda7907cb325","Type":"ContainerStarted","Data":"091689305d0e26df7ea1b3bd8fbdfbe4faa3a8e89a11bb65a28e0b41eeff4966"}
Sep 10 14:27:53.413774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.413696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal" event={"ID":"424c21ac2ec6f354644b2292aeae7f24","Type":"ContainerStarted","Data":"1d703351e7cc69486e0b83ed9fc6a9cc814130edd25fb1f9d0798801e8d83e24"}
Sep 10 14:27:53.418889 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.418799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw" event={"ID":"560dd0fe-d588-4a95-9882-4b8a0c64f1c2","Type":"ContainerStarted","Data":"4f69b048ce88295cfe69caa37d7256074a9035bae67e6278d5c61846dba4ebd4"}
Sep 10 14:27:53.423728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.423649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal" event={"ID":"c36c54bc5a50414caa33ecd6875e0504","Type":"ContainerStarted","Data":"509daa38a3b11d766fc9bf7db5a28318f0f0e57fc4910e3c67193bc60ccd02b6"}
Sep 10 14:27:53.426845 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.426778    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"349aec116818a749098bf0b5c6f16c1550a96c6987dee158c8f508027622cda7"}
Sep 10 14:27:53.734053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.733319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:53.734053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.733380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:53.734053 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.733518    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:53.734053 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.733581    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:55.733562539 +0000 UTC m=+5.089375429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:53.734053 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.733969    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:53.734053 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.734016    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:55.734000332 +0000 UTC m=+5.089813232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:53.834014 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:53.833977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:53.834197 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.834163    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:27:53.834197 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.834188    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:27:53.834350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.834202    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:53.834350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:53.834267    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:55.834248211 +0000 UTC m=+5.190061121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:54.247991 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:54.247946    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:22:52 +0000 UTC" deadline="2028-04-30 09:30:19.228258019 +0000 UTC"
Sep 10 14:27:54.247991 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:54.247990    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14347h2m24.980272571s"
Sep 10 14:27:54.312517 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:54.312486    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:54.312695 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:54.312642    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:27:54.313070 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:54.313046    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:54.313173 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:54.313150    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:27:54.663265 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:54.663184    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:27:55.313437 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:55.313404    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:55.313899 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.313532    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:27:55.757345 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:55.757305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:55.757554 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:55.757366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:55.757554 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.757527    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:55.757669 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.757594    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:59.75757353 +0000 UTC m=+9.113386439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:55.758030 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.758010    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:55.758114 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.758071    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:59.758054791 +0000 UTC m=+9.113867694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:55.858219 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:55.858181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:55.858414 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.858378    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:27:55.858414 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.858397    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:27:55.858414 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.858413    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:55.858573 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:55.858477    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:27:59.858457051 +0000 UTC m=+9.214270007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:56.313441 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:56.312732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:56.313441 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:56.312869    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:27:56.313441 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:56.313270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:56.313441 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:56.313391    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:27:57.314324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:57.312805    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:57.314324 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:57.312929    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:27:58.312326 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:58.312210    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:58.312500 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:58.312440    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:27:58.312500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:58.312472    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:58.312593 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:58.312579    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:27:59.312746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:59.312711    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:59.313217 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.312852    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:27:59.794001 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:59.793953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:27:59.794157 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:59.794018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:27:59.794232 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.794143    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:59.794232 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.794206    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:59.794364 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.794232    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:07.794210324 +0000 UTC m=+17.150023224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:27:59.794364 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.794361    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:07.79434385 +0000 UTC m=+17.150156739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:27:59.894565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:27:59.894457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:27:59.894737 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.894629    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:27:59.894737 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.894654    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:27:59.894737 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.894667    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:27:59.894737 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:27:59.894737    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:07.894716416 +0000 UTC m=+17.250529320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:28:00.312589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:00.312476    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:00.312589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:00.312530    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:00.313225 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:00.312628    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:00.313225 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:00.312753    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:01.313976 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:01.313938    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:01.314409 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:01.314068    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:02.312714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:02.312680    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:02.312903 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:02.312800    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:02.312903 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:02.312849    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:02.313014 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:02.312971    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:03.312867 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:03.312832    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:03.313355 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:03.312957    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:04.312853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:04.312810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:04.312853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:04.312849    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:04.313391 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:04.312949    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:04.313391 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:04.313075    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:05.312216 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:05.312169    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:05.312429 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:05.312339    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:06.312203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:06.312168    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:06.312781 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:06.312176    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:06.312781 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:06.312309    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:06.312781 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:06.312415    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:07.312835 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:07.312796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:07.313302 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.312909    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:07.854717 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:07.854679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:07.854886 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:07.854770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:07.854886 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.854841    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:28:07.854886 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.854850    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:28:07.855031 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.854910    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:23.854893369 +0000 UTC m=+33.210706258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:28:07.855031 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.854929    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:23.854918882 +0000 UTC m=+33.210731766 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:28:07.955537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:07.955500    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:07.955704 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.955642    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:28:07.955704 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.955671    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:28:07.955704 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.955683    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:28:07.955816 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:07.955737    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:23.955724026 +0000 UTC m=+33.311536927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:28:08.312865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:08.312829    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:08.312865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:08.312853    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:08.313462 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:08.312953    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:08.313462 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:08.313086    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:09.312193 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:09.312152    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:09.312401 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:09.312303    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:10.312686 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:10.312651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:10.313111 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:10.312651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:10.313111 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:10.312790    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:10.313111 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:10.312878    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:11.317080 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:11.317054    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:11.317471 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:11.317158    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:11.500389 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:11.500145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal" event={"ID":"424c21ac2ec6f354644b2292aeae7f24","Type":"ContainerStarted","Data":"1cabb63b18bad5f029170cdd192d1872982e7e612f1a645b710e31180b46b530"}
Sep 10 14:28:11.503034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:11.503008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lzvz4" event={"ID":"d9ec070a-311b-45f2-8375-8f94b0d9a69b","Type":"ContainerStarted","Data":"feb0414bb103fb30049b2fe18fdfd45cecd4b2d78e2b2ebf9561d21fe0dc1f40"}
Sep 10 14:28:11.529776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:11.529726    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-253.ec2.internal" podStartSLOduration=20.52971189 podStartE2EDuration="20.52971189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:11.513578 +0000 UTC m=+20.869390906" watchObservedRunningTime="2026-09-10 14:28:11.52971189 +0000 UTC m=+20.885524844"
Sep 10 14:28:11.529891 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:11.529851    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lzvz4" podStartSLOduration=1.68772323 podStartE2EDuration="20.529845836s" totalImagesPullingTime="18.842122606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:11.529692165 +0000 UTC m=+20.885505073" watchObservedRunningTime="2026-09-10 14:28:11.529845836 +0000 UTC m=+20.885658744"
Sep 10 14:28:12.312148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.312123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:12.312243 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:12.312220    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:12.312319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.312287    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:12.312430 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:12.312411    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:12.506294 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.506218    2560 generic.go:356] "Generic (PLEG): container finished" podID="53311511-e278-4666-bb1f-30c6c101fede" containerID="f973fb7e0d4ec24ce0ab85c98326db7d114ec5888a79a7b1c86f9d94d3519448" exitCode=0
Sep 10 14:28:12.507109 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.506310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerDied","Data":"f973fb7e0d4ec24ce0ab85c98326db7d114ec5888a79a7b1c86f9d94d3519448"}
Sep 10 14:28:12.507832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.507802    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwfht" event={"ID":"8d7361c0-0247-4988-bd32-fda7907cb325","Type":"ContainerStarted","Data":"d62a09764ab42f88778fa090ce8b2051f256eb8484db3d4c9bf44907d09c8a60"}
Sep 10 14:28:12.509502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.509479    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw" event={"ID":"560dd0fe-d588-4a95-9882-4b8a0c64f1c2","Type":"ContainerStarted","Data":"0aeaa0fd6d5321664037a5d75cbe976a235327c05679128531c4f6bf2f29279a"}
Sep 10 14:28:12.510898 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.510879    2560 generic.go:356] "Generic (PLEG): container finished" podID="c36c54bc5a50414caa33ecd6875e0504" containerID="5fdfe64eeeccfa6eeda79b810f831bf244e418df5da40ad672ead4a27aa30b81" exitCode=0
Sep 10 14:28:12.510976 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.510934    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal" event={"ID":"c36c54bc5a50414caa33ecd6875e0504","Type":"ContainerDied","Data":"5fdfe64eeeccfa6eeda79b810f831bf244e418df5da40ad672ead4a27aa30b81"}
Sep 10 14:28:12.514380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.514361    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"db16d58fb0637c2b99c16344d6a6dc936e48c71286f9f4e1c31d7f16a38dd4f7"}
Sep 10 14:28:12.514467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.514417    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"29216571ba9d3863fe0c2b0c566c3b2a4a998369c93eaf2759eea4f94f9fc1a0"}
Sep 10 14:28:12.514467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.514463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"7c5632930ef8fb12d5300f9138cd3c2f48ff397c460f0a17214249cf009d45f6"}
Sep 10 14:28:12.514582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.514495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"137be92aeade0d0608a94b24027949743b9a119c5158fc65f78ae7c81219a51a"}
Sep 10 14:28:12.514582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.514521    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"e504c42f617f0035028790f5787965e3612e796221a0d925328466e7a16823f6"}
Sep 10 14:28:12.514582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.514554    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"59570a850be399e8d25fc0bb52ab09bf823dabc4a8e3480f7432654bcb1db519"}
Sep 10 14:28:12.515755 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.515723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d8sfz" event={"ID":"2db006d1-41d7-487e-a3ee-ded18e3db48a","Type":"ContainerStarted","Data":"fbe5aad20fc597987ed7680c136af4b6979b0582c62effea33aa18d67e0f6eea"}
Sep 10 14:28:12.517067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.517047    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bbmm" event={"ID":"a2e559fb-9b78-4509-b544-3410d402bda8","Type":"ContainerStarted","Data":"90538cda1492113efab14ddb19ac7b5895ff967f55ef64aa91685f06bbcf0049"}
Sep 10 14:28:12.518384 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.518365    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x5vpk" event={"ID":"d25fd70d-c6c7-46fb-80bd-43227ad3c20c","Type":"ContainerStarted","Data":"6691d77ec79e35fc3b7d957b7e2577ec8a99f3f030a43b50f28d7019a47c593b"}
Sep 10 14:28:12.519885 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.519861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pksmv" event={"ID":"6cddba93-d2fc-4e91-be06-a8559e276d53","Type":"ContainerStarted","Data":"17cf5d72d032b975dc7af8321915c6c1f464c63aa82e7469578a53a62143a46b"}
Sep 10 14:28:12.550864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.550812    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6bbmm" podStartSLOduration=3.260136766 podStartE2EDuration="21.550792867s" totalImagesPullingTime="18.290656101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:12.550544263 +0000 UTC m=+21.906357188" watchObservedRunningTime="2026-09-10 14:28:12.550792867 +0000 UTC m=+21.906605773"
Sep 10 14:28:12.565795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.565745    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d8sfz" podStartSLOduration=2.730497674 podStartE2EDuration="21.565727112s" totalImagesPullingTime="18.835229438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:12.565719352 +0000 UTC m=+21.921532282" watchObservedRunningTime="2026-09-10 14:28:12.565727112 +0000 UTC m=+21.921540034"
Sep 10 14:28:12.578995 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.578959    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pksmv" podStartSLOduration=2.811900987 podStartE2EDuration="21.578945271s" totalImagesPullingTime="18.767044284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:12.578473715 +0000 UTC m=+21.934286655" watchObservedRunningTime="2026-09-10 14:28:12.578945271 +0000 UTC m=+21.934758175"
Sep 10 14:28:12.592067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.592028    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lwfht" podStartSLOduration=2.768075701 podStartE2EDuration="21.592014736s" totalImagesPullingTime="18.823939035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:12.591579664 +0000 UTC m=+21.947392590" watchObservedRunningTime="2026-09-10 14:28:12.592014736 +0000 UTC m=+21.947827645"
Sep 10 14:28:12.604936 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:12.604895    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x5vpk" podStartSLOduration=2.863035453 podStartE2EDuration="21.604882314s" totalImagesPullingTime="18.741846861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:12.604590501 +0000 UTC m=+21.960403408" watchObservedRunningTime="2026-09-10 14:28:12.604882314 +0000 UTC m=+21.960695221"
Sep 10 14:28:13.312830 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.312799    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:13.313076 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:13.312934    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:13.528143 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.528097    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal" event={"ID":"c36c54bc5a50414caa33ecd6875e0504","Type":"ContainerStarted","Data":"336706b5985742cc244d7a6b83b090368a581c0990ea785112be44229e8f8ed6"}
Sep 10 14:28:13.543473 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.543420    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-253.ec2.internal" podStartSLOduration=22.543402301 podStartE2EDuration="22.543402301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:13.542982339 +0000 UTC m=+22.898795246" watchObservedRunningTime="2026-09-10 14:28:13.543402301 +0000 UTC m=+22.899215210"
Sep 10 14:28:13.583227 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.583148    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:28:13.584185 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.584159    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:28:13.714084 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.714051    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:28:13.714620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.714605    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6bbmm"
Sep 10 14:28:13.768455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:13.768250    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 14:28:14.276574 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:14.276460    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T14:28:13.768449728Z","UUID":"e594d2da-ce5c-48e2-9fff-df82f9fe21a6","Handler":null,"Name":"","Endpoint":""}
Sep 10 14:28:14.278567 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:14.278544    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 14:28:14.278711 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:14.278579    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 14:28:14.312730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:14.312703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:14.312901 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:14.312816    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:14.312901 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:14.312861    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:14.313016 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:14.312968    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:14.533147 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:14.533035    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw" event={"ID":"560dd0fe-d588-4a95-9882-4b8a0c64f1c2","Type":"ContainerStarted","Data":"0d38cd3cca4c90c153408461131dddfba718ed3b424a6b9ff91b8c655a65db6e"}
Sep 10 14:28:15.311971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:15.311938    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:15.312116 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:15.312062    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:15.537450 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:15.537414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw" event={"ID":"560dd0fe-d588-4a95-9882-4b8a0c64f1c2","Type":"ContainerStarted","Data":"cc503d3a1b05a2bc71e6d745aa4a3f6f6eaff9aaa8d51623bd0bd7f5eab9f2bd"}
Sep 10 14:28:15.540824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:15.540799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"35c73abc81bbcea08de40fc12ac5c9f4c7e52c37c7d01d6e933bc363f894372c"}
Sep 10 14:28:15.558081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:15.557038    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k26fw" podStartSLOduration=3.113134338 podStartE2EDuration="24.5570196s" totalImagesPullingTime="21.443885262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:15.556542995 +0000 UTC m=+24.912355926" watchObservedRunningTime="2026-09-10 14:28:15.5570196 +0000 UTC m=+24.912832508"
Sep 10 14:28:16.312026 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:16.311992    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:16.312186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:16.312039    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:16.312186 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:16.312121    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:16.312298 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:16.312256    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:17.312432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:17.312356    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:17.312718 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:17.312478    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:17.550828 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:17.550793    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" event={"ID":"e289bd29-96ce-4777-b205-7a700842368d","Type":"ContainerStarted","Data":"b10de8cf4bd33c3fa59186300c34547400c0e4f72dec05c9f684beaf5a9b8ddf"}
Sep 10 14:28:17.551089 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:17.551063    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:28:17.551219 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:17.551162    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:28:17.568731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:17.568516    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:28:17.577369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:17.577332    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq" podStartSLOduration=7.390291419 podStartE2EDuration="26.577318902s" totalImagesPullingTime="19.187027483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:17.576913148 +0000 UTC m=+26.932726054" watchObservedRunningTime="2026-09-10 14:28:17.577318902 +0000 UTC m=+26.933131808"
Sep 10 14:28:18.312541 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:18.312504    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:18.313024 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:18.312618    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:18.313024 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:18.312665    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:18.313024 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:18.312757    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:18.557199 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:18.557166    2560 generic.go:356] "Generic (PLEG): container finished" podID="53311511-e278-4666-bb1f-30c6c101fede" containerID="93aa50c50c581490b5a09913b1fbbafe6f3bdeecee2fa26fd2013b751700c961" exitCode=0
Sep 10 14:28:18.558054 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:18.557841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerDied","Data":"93aa50c50c581490b5a09913b1fbbafe6f3bdeecee2fa26fd2013b751700c961"}
Sep 10 14:28:18.559484 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:18.558662    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:28:18.583195 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:18.583171    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:28:19.164069 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:19.164035    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ndntw"]
Sep 10 14:28:19.164363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:19.164181    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:19.164363 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:19.164301    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:19.166597 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:19.166569    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-57lnd"]
Sep 10 14:28:19.166727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:19.166675    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:19.166790 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:19.166771    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:19.175329 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:19.175303    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6v7j"]
Sep 10 14:28:19.175446 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:19.175397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:19.175513 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:19.175485    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:20.312984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:20.312946    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:20.312984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:20.312979    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:20.313644 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:20.313069    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:20.313644 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:20.313121    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:20.313644 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:20.313206    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:20.313644 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:20.313302    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:20.564003 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:20.563923    2560 generic.go:356] "Generic (PLEG): container finished" podID="53311511-e278-4666-bb1f-30c6c101fede" containerID="b8efab74c4e782c368cf93886b6d3b0375e7be9b4512119f61ab4939814da4a8" exitCode=0
Sep 10 14:28:20.565306 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:20.564208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerDied","Data":"b8efab74c4e782c368cf93886b6d3b0375e7be9b4512119f61ab4939814da4a8"}
Sep 10 14:28:21.568835 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:21.568747    2560 generic.go:356] "Generic (PLEG): container finished" podID="53311511-e278-4666-bb1f-30c6c101fede" containerID="936fbacc50bf59a724abaf75fceaf8ae7959718c487890fdc40569c24d3b4115" exitCode=0
Sep 10 14:28:21.568835 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:21.568812    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerDied","Data":"936fbacc50bf59a724abaf75fceaf8ae7959718c487890fdc40569c24d3b4115"}
Sep 10 14:28:22.312971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:22.312936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:22.313164 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:22.312936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:22.313164 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:22.313058    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:22.313164 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:22.313112    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:22.313164 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:22.312940    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:22.313376 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:22.313189    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:23.866538 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:23.866499    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:23.866992 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:23.866554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:23.866992 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.866635    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:28:23.866992 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.866660    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:28:23.866992 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.866722    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs podName:d0a7d218-c488-478d-ad91-3e119f318ad0 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:55.866700752 +0000 UTC m=+65.222513652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs") pod "network-metrics-daemon-57lnd" (UID: "d0a7d218-c488-478d-ad91-3e119f318ad0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:28:23.866992 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.866744    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret podName:807de0e5-36be-4155-b273-fc7eb43f2f53 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:55.866734396 +0000 UTC m=+65.222547282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret") pod "global-pull-secret-syncer-ndntw" (UID: "807de0e5-36be-4155-b273-fc7eb43f2f53") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:28:23.967095 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:23.967066    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:23.967245 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.967202    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:28:23.967245 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.967221    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:28:23.967245 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.967229    2560 projected.go:196] Error preparing data for projected volume kube-api-access-rps8h for pod openshift-network-diagnostics/network-check-target-c6v7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:28:23.967374 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:23.967294    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h podName:4ba171d4-5505-46f1-a8b7-95ce8474fcc0 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:55.967263385 +0000 UTC m=+65.323076282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rps8h" (UniqueName: "kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h") pod "network-check-target-c6v7j" (UID: "4ba171d4-5505-46f1-a8b7-95ce8474fcc0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:28:24.312497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.312464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:24.312657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.312543    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:24.312707 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.312656    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:24.312707 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.312672    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndntw" podUID="807de0e5-36be-4155-b273-fc7eb43f2f53"
Sep 10 14:28:24.312798 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.312780    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-57lnd" podUID="d0a7d218-c488-478d-ad91-3e119f318ad0"
Sep 10 14:28:24.312917 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.312895    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6v7j" podUID="4ba171d4-5505-46f1-a8b7-95ce8474fcc0"
Sep 10 14:28:24.454440 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.454416    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-253.ec2.internal" event="NodeReady"
Sep 10 14:28:24.454598 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.454537    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 14:28:24.487174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.487143    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7554c96755-ztg9w"]
Sep 10 14:28:24.525420 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.525391    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7554c96755-ztg9w"]
Sep 10 14:28:24.525566 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.525434    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-78xpw"]
Sep 10 14:28:24.525566 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.525501    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.527726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.527704    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2kwcf\""
Sep 10 14:28:24.527726 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.527704    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 14:28:24.527897 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.527704    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 14:28:24.527990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.527978    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 14:28:24.532803 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.532784    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 14:28:24.536860 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.536843    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6sxkb"]
Sep 10 14:28:24.536962 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.536948    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.539228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.539191    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 14:28:24.539228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.539218    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lsf8q\""
Sep 10 14:28:24.539416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.539229    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 14:28:24.549350 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.549330    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6sxkb"]
Sep 10 14:28:24.549446 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.549365    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-78xpw"]
Sep 10 14:28:24.549487 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.549465    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.551589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.551572    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 14:28:24.551774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.551758    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 14:28:24.551857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.551787    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z67dn\""
Sep 10 14:28:24.551857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.551797    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 14:28:24.641129 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.641095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c25814-e802-4f67-95cd-12a0a7fcd941-tmp-dir\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-installation-pull-secrets\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7c25814-e802-4f67-95cd-12a0a7fcd941-config-volume\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-image-registry-private-configuration\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671407    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4febf65c-8232-42e4-8e9e-89dfb9caa7db-ca-trust-extracted\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671449    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671473    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rphtb\" (UniqueName: \"kubernetes.io/projected/c7c25814-e802-4f67-95cd-12a0a7fcd941-kube-api-access-rphtb\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.671883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671531    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4e1db6f-fa90-47e4-86eb-23713483bc05-cert\") pod \"ingress-canary-6sxkb\" (UID: \"e4e1db6f-fa90-47e4-86eb-23713483bc05\") " pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.671883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-certificates\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-trusted-ca\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-bound-sa-token\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671687    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvnj2\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-kube-api-access-wvnj2\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.671883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.671730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs7pk\" (UniqueName: \"kubernetes.io/projected/e4e1db6f-fa90-47e4-86eb-23713483bc05-kube-api-access-cs7pk\") pod \"ingress-canary-6sxkb\" (UID: \"e4e1db6f-fa90-47e4-86eb-23713483bc05\") " pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.724171 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.724085    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:24.772643 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772581    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7c25814-e802-4f67-95cd-12a0a7fcd941-config-volume\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772652    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-image-registry-private-configuration\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4febf65c-8232-42e4-8e9e-89dfb9caa7db-ca-trust-extracted\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772759    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rphtb\" (UniqueName: \"kubernetes.io/projected/c7c25814-e802-4f67-95cd-12a0a7fcd941-kube-api-access-rphtb\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.772796    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4e1db6f-fa90-47e4-86eb-23713483bc05-cert\") pod \"ingress-canary-6sxkb\" (UID: \"e4e1db6f-fa90-47e4-86eb-23713483bc05\") " pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.772836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772833    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-certificates\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.772861    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls podName:c7c25814-e802-4f67-95cd-12a0a7fcd941 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:25.272841846 +0000 UTC m=+34.628654750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls") pod "dns-default-78xpw" (UID: "c7c25814-e802-4f67-95cd-12a0a7fcd941") : secret "dns-default-metrics-tls" not found
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-trusted-ca\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-bound-sa-token\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvnj2\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-kube-api-access-wvnj2\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.772965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs7pk\" (UniqueName: \"kubernetes.io/projected/e4e1db6f-fa90-47e4-86eb-23713483bc05-kube-api-access-cs7pk\") pod \"ingress-canary-6sxkb\" (UID: \"e4e1db6f-fa90-47e4-86eb-23713483bc05\") " pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.773014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c25814-e802-4f67-95cd-12a0a7fcd941-tmp-dir\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.773224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.773038    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-installation-pull-secrets\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773581 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.773262    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4febf65c-8232-42e4-8e9e-89dfb9caa7db-ca-trust-extracted\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773581 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.773275    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7c25814-e802-4f67-95cd-12a0a7fcd941-config-volume\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.773581 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.773438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-certificates\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.773581 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.773466    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:28:24.773581 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.773485    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7554c96755-ztg9w: secret "image-registry-tls" not found
Sep 10 14:28:24.773581 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:24.773543    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls podName:4febf65c-8232-42e4-8e9e-89dfb9caa7db nodeName:}" failed. No retries permitted until 2026-09-10 14:28:25.27352572 +0000 UTC m=+34.629338614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls") pod "image-registry-7554c96755-ztg9w" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db") : secret "image-registry-tls" not found
Sep 10 14:28:24.773956 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.773935    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c25814-e802-4f67-95cd-12a0a7fcd941-tmp-dir\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.774215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.774167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-trusted-ca\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.777251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.777232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-image-registry-private-configuration\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.777360 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.777235    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-installation-pull-secrets\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.777360 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.777266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4e1db6f-fa90-47e4-86eb-23713483bc05-cert\") pod \"ingress-canary-6sxkb\" (UID: \"e4e1db6f-fa90-47e4-86eb-23713483bc05\") " pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.782600 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.782574    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-bound-sa-token\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.782763 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.782746    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs7pk\" (UniqueName: \"kubernetes.io/projected/e4e1db6f-fa90-47e4-86eb-23713483bc05-kube-api-access-cs7pk\") pod \"ingress-canary-6sxkb\" (UID: \"e4e1db6f-fa90-47e4-86eb-23713483bc05\") " pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:24.782849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.782767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvnj2\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-kube-api-access-wvnj2\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:24.783141 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.783126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rphtb\" (UniqueName: \"kubernetes.io/projected/c7c25814-e802-4f67-95cd-12a0a7fcd941-kube-api-access-rphtb\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:24.857120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:24.857082    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6sxkb"
Sep 10 14:28:25.277822 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:25.277786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:25.278350 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:25.277852    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:25.278350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:25.277981    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:28:25.278350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:25.278001    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7554c96755-ztg9w: secret "image-registry-tls" not found
Sep 10 14:28:25.278350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:25.277980    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:28:25.278350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:25.278067    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls podName:4febf65c-8232-42e4-8e9e-89dfb9caa7db nodeName:}" failed. No retries permitted until 2026-09-10 14:28:26.278047515 +0000 UTC m=+35.633860400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls") pod "image-registry-7554c96755-ztg9w" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db") : secret "image-registry-tls" not found
Sep 10 14:28:25.278350 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:25.278087    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls podName:c7c25814-e802-4f67-95cd-12a0a7fcd941 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:26.278077975 +0000 UTC m=+35.633890869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls") pod "dns-default-78xpw" (UID: "c7c25814-e802-4f67-95cd-12a0a7fcd941") : secret "dns-default-metrics-tls" not found
Sep 10 14:28:26.285107 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.284910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:26.285473 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:26.285053    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:28:26.285473 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.285181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:26.285473 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:26.285189    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7554c96755-ztg9w: secret "image-registry-tls" not found
Sep 10 14:28:26.285473 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:26.285247    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls podName:4febf65c-8232-42e4-8e9e-89dfb9caa7db nodeName:}" failed. No retries permitted until 2026-09-10 14:28:28.285229082 +0000 UTC m=+37.641041972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls") pod "image-registry-7554c96755-ztg9w" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db") : secret "image-registry-tls" not found
Sep 10 14:28:26.285473 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:26.285316    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:28:26.285473 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:26.285361    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls podName:c7c25814-e802-4f67-95cd-12a0a7fcd941 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:28.285348279 +0000 UTC m=+37.641161178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls") pod "dns-default-78xpw" (UID: "c7c25814-e802-4f67-95cd-12a0a7fcd941") : secret "dns-default-metrics-tls" not found
Sep 10 14:28:26.312843 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.312802    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:26.313019 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.312803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:26.313019 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.312807    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:26.316084 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.316059    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:28:26.316231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.316150    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bf25v\""
Sep 10 14:28:26.316231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.316166    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:28:26.316231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.316155    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pptck\""
Sep 10 14:28:26.316231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.316206    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:28:26.316442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.316168    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:28:26.415382 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.415347    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6sxkb"]
Sep 10 14:28:26.420657 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:26.420632    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e1db6f_fa90_47e4_86eb_23713483bc05.slice/crio-35670d10e3e86134e0826f92de0f8c6f4bf915090bbcf2a9b317a33264b6b951 WatchSource:0}: Error finding container 35670d10e3e86134e0826f92de0f8c6f4bf915090bbcf2a9b317a33264b6b951: Status 404 returned error can't find the container with id 35670d10e3e86134e0826f92de0f8c6f4bf915090bbcf2a9b317a33264b6b951
Sep 10 14:28:26.587534 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.587494    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6sxkb" event={"ID":"e4e1db6f-fa90-47e4-86eb-23713483bc05","Type":"ContainerStarted","Data":"35670d10e3e86134e0826f92de0f8c6f4bf915090bbcf2a9b317a33264b6b951"}
Sep 10 14:28:26.590738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:26.590705    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerStarted","Data":"10f9df4eebbfeda457d5a6799dd31976042c2402af40c2706bf821162384377f"}
Sep 10 14:28:27.155418 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.155384    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.246944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.246908    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.347468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.347433    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.493949 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.493917    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.550407 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.550374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.596645 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.596615    2560 generic.go:356] "Generic (PLEG): container finished" podID="53311511-e278-4666-bb1f-30c6c101fede" containerID="10f9df4eebbfeda457d5a6799dd31976042c2402af40c2706bf821162384377f" exitCode=0
Sep 10 14:28:27.596814 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:27.596654    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerDied","Data":"10f9df4eebbfeda457d5a6799dd31976042c2402af40c2706bf821162384377f"}
Sep 10 14:28:28.299912 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:28.299874    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:28.300070 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:28.299958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:28.300070 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:28.300053    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:28:28.300183 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:28.300111    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:28:28.300183 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:28.300132    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls podName:c7c25814-e802-4f67-95cd-12a0a7fcd941 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:32.300109091 +0000 UTC m=+41.655922019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls") pod "dns-default-78xpw" (UID: "c7c25814-e802-4f67-95cd-12a0a7fcd941") : secret "dns-default-metrics-tls" not found
Sep 10 14:28:28.300183 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:28.300132    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7554c96755-ztg9w: secret "image-registry-tls" not found
Sep 10 14:28:28.300183 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:28.300172    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls podName:4febf65c-8232-42e4-8e9e-89dfb9caa7db nodeName:}" failed. No retries permitted until 2026-09-10 14:28:32.300161884 +0000 UTC m=+41.655974776 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls") pod "image-registry-7554c96755-ztg9w" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db") : secret "image-registry-tls" not found
Sep 10 14:28:28.604868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:28.604786    2560 generic.go:356] "Generic (PLEG): container finished" podID="53311511-e278-4666-bb1f-30c6c101fede" containerID="482d4c028ea514a3bb0b3aae9d2d39a76fd3932be9869ea73f5ab6a4b1edc004" exitCode=0
Sep 10 14:28:28.604868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:28.604818    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerDied","Data":"482d4c028ea514a3bb0b3aae9d2d39a76fd3932be9869ea73f5ab6a4b1edc004"}
Sep 10 14:28:29.241921 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.241890    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:29.305401 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.305342    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:29.496167 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.496130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:29.596338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.596251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:29.608505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.608450    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6sxkb" event={"ID":"e4e1db6f-fa90-47e4-86eb-23713483bc05","Type":"ContainerStarted","Data":"d7e875375ec5b69eb840c603baa48d17131f5b1f7e195fde9d9d0f305e29173f"}
Sep 10 14:28:29.611994 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.611973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" event={"ID":"53311511-e278-4666-bb1f-30c6c101fede","Type":"ContainerStarted","Data":"2b8e0ddde949391781a90d2bfd9261a93e8c64bab700fead944ec22ea21f0b24"}
Sep 10 14:28:29.623833 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.623794    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6sxkb" podStartSLOduration=2.694265235 podStartE2EDuration="5.623780151s" totalImagesPullingTime="2.929514916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:29.622836645 +0000 UTC m=+38.978649582" watchObservedRunningTime="2026-09-10 14:28:29.623780151 +0000 UTC m=+38.979593085"
Sep 10 14:28:29.642680 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.642634    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6bhrh" podStartSLOduration=8.231936432 podStartE2EDuration="38.642597481s" totalImagesPullingTime="30.410661049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:29.641622638 +0000 UTC m=+38.997435544" watchObservedRunningTime="2026-09-10 14:28:29.642597481 +0000 UTC m=+38.998410387"
Sep 10 14:28:29.950432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.950272    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-lm8pr"]
Sep 10 14:28:29.971482 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.971017    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-5sv6n"]
Sep 10 14:28:29.971482 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.971399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr"
Sep 10 14:28:29.973742 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.973720    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cmz5j\""
Sep 10 14:28:29.981259 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.981241    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-758b49888b-qhk2k"]
Sep 10 14:28:29.981389 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.981374    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:29.983551 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.983531    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 14:28:29.983710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.983641    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 14:28:29.983805 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.983789    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-74pd6\""
Sep 10 14:28:29.983944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.983929    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 14:28:29.984025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.984012    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 14:28:29.989055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.988889    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 14:28:29.994348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.994330    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-5sv6n"]
Sep 10 14:28:29.994445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.994358    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-lm8pr"]
Sep 10 14:28:29.994445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.994372    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-758b49888b-qhk2k"]
Sep 10 14:28:29.994548 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.994458    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:29.996641 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.996625    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 14:28:29.996733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.996650    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 14:28:29.996733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.996702    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 14:28:29.996733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.996715    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 14:28:29.996942 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.996928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 14:28:29.997103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.997087    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v8qsc\""
Sep 10 14:28:29.997171 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:29.997123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 14:28:30.115245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115212    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-default-certificate\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.115245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mvgj\" (UniqueName: \"kubernetes.io/projected/8343c732-0f02-49d0-b506-c04b91e1e990-kube-api-access-8mvgj\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.115502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115265    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-tmp\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.115502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.115502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.115502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r887c\" (UniqueName: \"kubernetes.io/projected/84838535-30a3-4238-8f8b-0e2fe2e33072-kube-api-access-r887c\") pod \"network-check-source-685456655f-lm8pr\" (UID: \"84838535-30a3-4238-8f8b-0e2fe2e33072\") " pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr"
Sep 10 14:28:30.115502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115468    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.115744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-snapshots\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.115744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115624    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.115744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdzlq\" (UniqueName: \"kubernetes.io/projected/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-kube-api-access-pdzlq\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.115744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115680    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-serving-cert\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.115744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.115705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-stats-auth\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.140384 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.140353    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:30.216468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.216468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-snapshots\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.216698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.216698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdzlq\" (UniqueName: \"kubernetes.io/projected/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-kube-api-access-pdzlq\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.216698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-serving-cert\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.216698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216669    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-stats-auth\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.216880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-default-certificate\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.216880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mvgj\" (UniqueName: \"kubernetes.io/projected/8343c732-0f02-49d0-b506-c04b91e1e990-kube-api-access-8mvgj\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.216880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216824    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-tmp\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.216880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216859    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r887c\" (UniqueName: \"kubernetes.io/projected/84838535-30a3-4238-8f8b-0e2fe2e33072-kube-api-access-r887c\") pod \"network-check-source-685456655f-lm8pr\" (UID: \"84838535-30a3-4238-8f8b-0e2fe2e33072\") " pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr"
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.216934    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-snapshots\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.217024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:30.217044    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:30.217048    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:30.71703227 +0000 UTC m=+40.072845173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : configmap references non-existent config key: service-ca.crt
Sep 10 14:28:30.217166 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:30.217107    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:30.717090488 +0000 UTC m=+40.072903378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : secret "router-metrics-certs-default" not found
Sep 10 14:28:30.217552 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.217397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-tmp\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.217975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.217958    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.219932 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.219909    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-stats-auth\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.220024 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.219989    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-serving-cert\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.220118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.220101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-default-certificate\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.226843 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.226789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdzlq\" (UniqueName: \"kubernetes.io/projected/1d2a0a89-de95-498a-b00e-7ad8ecc0e906-kube-api-access-pdzlq\") pod \"insights-operator-76bcd6ccf5-5sv6n\" (UID: \"1d2a0a89-de95-498a-b00e-7ad8ecc0e906\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.227434 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.227415    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r887c\" (UniqueName: \"kubernetes.io/projected/84838535-30a3-4238-8f8b-0e2fe2e33072-kube-api-access-r887c\") pod \"network-check-source-685456655f-lm8pr\" (UID: \"84838535-30a3-4238-8f8b-0e2fe2e33072\") " pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr"
Sep 10 14:28:30.227688 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.227671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mvgj\" (UniqueName: \"kubernetes.io/projected/8343c732-0f02-49d0-b506-c04b91e1e990-kube-api-access-8mvgj\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.231644 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.231621    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:30.280925 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.280895    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr"
Sep 10 14:28:30.290207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.290187    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n"
Sep 10 14:28:30.583041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.582920    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-lm8pr"]
Sep 10 14:28:30.585252 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:30.585214    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84838535_30a3_4238_8f8b_0e2fe2e33072.slice/crio-ffa326006e41a24bdf7438b1b39e004d086c6df2074e9c2fb4508c21c91ce2ec WatchSource:0}: Error finding container ffa326006e41a24bdf7438b1b39e004d086c6df2074e9c2fb4508c21c91ce2ec: Status 404 returned error can't find the container with id ffa326006e41a24bdf7438b1b39e004d086c6df2074e9c2fb4508c21c91ce2ec
Sep 10 14:28:30.595884 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.595856    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:30.609322 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.609298    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-5sv6n"]
Sep 10 14:28:30.611927 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:30.611898    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2a0a89_de95_498a_b00e_7ad8ecc0e906.slice/crio-754122d8fff7b4e512dea1eb9622c1c107871f5f30f10eb8aa9a0ad766819b92 WatchSource:0}: Error finding container 754122d8fff7b4e512dea1eb9622c1c107871f5f30f10eb8aa9a0ad766819b92: Status 404 returned error can't find the container with id 754122d8fff7b4e512dea1eb9622c1c107871f5f30f10eb8aa9a0ad766819b92
Sep 10 14:28:30.615702 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.615668    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n" event={"ID":"1d2a0a89-de95-498a-b00e-7ad8ecc0e906","Type":"ContainerStarted","Data":"754122d8fff7b4e512dea1eb9622c1c107871f5f30f10eb8aa9a0ad766819b92"}
Sep 10 14:28:30.616826 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.616807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr" event={"ID":"84838535-30a3-4238-8f8b-0e2fe2e33072","Type":"ContainerStarted","Data":"ffa326006e41a24bdf7438b1b39e004d086c6df2074e9c2fb4508c21c91ce2ec"}
Sep 10 14:28:30.696999 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.696968    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:30.721182 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.721148    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.721361 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:30.721191    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:30.721361 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:30.721298    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:31.721258958 +0000 UTC m=+41.077071852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : configmap references non-existent config key: service-ca.crt
Sep 10 14:28:30.721361 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:30.721342    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 14:28:30.721541 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:30.721405    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:31.721387094 +0000 UTC m=+41.077199993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : secret "router-metrics-certs-default" not found
Sep 10 14:28:31.730142 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:31.730104    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:31.730619 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:31.730148    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:31.730619 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:31.730300    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 14:28:31.730619 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:31.730320    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:33.73029041 +0000 UTC m=+43.086103312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : configmap references non-existent config key: service-ca.crt
Sep 10 14:28:31.730619 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:31.730368    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:33.730351574 +0000 UTC m=+43.086164459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : secret "router-metrics-certs-default" not found
Sep 10 14:28:32.336330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:32.336275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:32.336500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:32.336371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:32.336500 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:32.336397    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:28:32.336500 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:32.336469    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls podName:c7c25814-e802-4f67-95cd-12a0a7fcd941 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:40.336447672 +0000 UTC m=+49.692260575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls") pod "dns-default-78xpw" (UID: "c7c25814-e802-4f67-95cd-12a0a7fcd941") : secret "dns-default-metrics-tls" not found
Sep 10 14:28:32.336722 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:32.336530    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:28:32.336722 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:32.336547    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7554c96755-ztg9w: secret "image-registry-tls" not found
Sep 10 14:28:32.336722 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:32.336600    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls podName:4febf65c-8232-42e4-8e9e-89dfb9caa7db nodeName:}" failed. No retries permitted until 2026-09-10 14:28:40.336584454 +0000 UTC m=+49.692397352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls") pod "image-registry-7554c96755-ztg9w" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db") : secret "image-registry-tls" not found
Sep 10 14:28:32.908770 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:32.908731    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:32.965228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:32.965191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:33.750488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:33.750451    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:33.750488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:33.750487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:33.750733 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:33.750606    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 14:28:33.750733 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:33.750636    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:37.750617006 +0000 UTC m=+47.106429911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : configmap references non-existent config key: service-ca.crt
Sep 10 14:28:33.750733 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:33.750660    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:37.750653633 +0000 UTC m=+47.106466517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : secret "router-metrics-certs-default" not found
Sep 10 14:28:34.630151 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:34.630121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n" event={"ID":"1d2a0a89-de95-498a-b00e-7ad8ecc0e906","Type":"ContainerStarted","Data":"c92ca36505e0fe7a35cd9db761b368d369a85eb023b7529539aab3ef36928f42"}
Sep 10 14:28:34.646491 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:34.646449    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n" podStartSLOduration=2.0637822789999998 podStartE2EDuration="5.646430175s" totalImagesPullingTime="3.582647896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:34.645786099 +0000 UTC m=+44.001599030" watchObservedRunningTime="2026-09-10 14:28:34.646430175 +0000 UTC m=+44.002243082"
Sep 10 14:28:35.517754 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.517711    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6j84z"]
Sep 10 14:28:35.530604 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.530569    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:35.550493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.550449    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6j84z"]
Sep 10 14:28:35.550628 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.550576    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.552617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.552594    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 14:28:35.553010 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.552985    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 14:28:35.553125 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.553088    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fc42h\""
Sep 10 14:28:35.566380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.566353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-crio-socket\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.566480 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.566398    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-data-volume\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.566575 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.566546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc54c\" (UniqueName: \"kubernetes.io/projected/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-api-access-cc54c\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.566634 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.566589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.566705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.566686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.593798 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.593758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:35.634505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.634468    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr" event={"ID":"84838535-30a3-4238-8f8b-0e2fe2e33072","Type":"ContainerStarted","Data":"5017dad4d891f58b1dac41e7257ee95bf52f9ae6224f81d6eb964944f26786f9"}
Sep 10 14:28:35.649008 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.648960    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-lm8pr" podStartSLOduration=2.6763184129999997 podStartE2EDuration="6.648944272s" totalImagesPullingTime="3.972625859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:35.648422528 +0000 UTC m=+45.004235454" watchObservedRunningTime="2026-09-10 14:28:35.648944272 +0000 UTC m=+45.004757179"
Sep 10 14:28:35.667100 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc54c\" (UniqueName: \"kubernetes.io/projected/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-api-access-cc54c\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.667226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:35.667226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.667226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667150    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.667226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667202    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-crio-socket\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.667474 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:35.667232    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:28:35.667474 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667268    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-data-volume\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.667474 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:35.667304    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:35.667474 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:35.667315    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:36.167275502 +0000 UTC m=+45.523088404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : configmap "kube-rbac-proxy" not found
Sep 10 14:28:35.667474 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:35.667343    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:36.167332799 +0000 UTC m=+45.523145701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:35.667474 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-crio-socket\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.667790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.667563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-data-volume\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.677424 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.677401    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc54c\" (UniqueName: \"kubernetes.io/projected/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-api-access-cc54c\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:35.763664 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:35.763628    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:36.171046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.171011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:36.171270 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.171062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:36.171270 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:36.171150    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:28:36.171270 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:36.171181    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:36.171270 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:36.171240    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:37.171216384 +0000 UTC m=+46.527029268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : configmap "kube-rbac-proxy" not found
Sep 10 14:28:36.171270 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:36.171258    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:37.171249923 +0000 UTC m=+46.527062808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:36.368820 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.368786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:36.447509 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.447476    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:36.756134 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.756058    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:36.777100 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.777076    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"]
Sep 10 14:28:36.798134 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.798114    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"]
Sep 10 14:28:36.798240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.798197    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"
Sep 10 14:28:36.800413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.800391    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 14:28:36.800987 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.800969    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 14:28:36.801076 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.800975    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-p9296\""
Sep 10 14:28:36.825290 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.825255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:36.877201 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.877174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl5tj\" (UniqueName: \"kubernetes.io/projected/1bc46da6-b07e-4f27-b980-1acad1ba60b0-kube-api-access-tl5tj\") pod \"migrator-fdbb8fdff-6dpbx\" (UID: \"1bc46da6-b07e-4f27-b980-1acad1ba60b0\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"
Sep 10 14:28:36.927648 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.927579    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:36.978218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.978192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl5tj\" (UniqueName: \"kubernetes.io/projected/1bc46da6-b07e-4f27-b980-1acad1ba60b0-kube-api-access-tl5tj\") pod \"migrator-fdbb8fdff-6dpbx\" (UID: \"1bc46da6-b07e-4f27-b980-1acad1ba60b0\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"
Sep 10 14:28:36.985471 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:36.985446    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl5tj\" (UniqueName: \"kubernetes.io/projected/1bc46da6-b07e-4f27-b980-1acad1ba60b0-kube-api-access-tl5tj\") pod \"migrator-fdbb8fdff-6dpbx\" (UID: \"1bc46da6-b07e-4f27-b980-1acad1ba60b0\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"
Sep 10 14:28:37.056459 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.056381    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:37.089052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.089029    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwfht_8d7361c0-0247-4988-bd32-fda7907cb325/dns-node-resolver/0.log"
Sep 10 14:28:37.107165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.107143    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"
Sep 10 14:28:37.160126 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.160094    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:37.179953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.179927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:37.180135 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.180112    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:37.180197 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.180136    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:37.180253 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.180204    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:39.180180788 +0000 UTC m=+48.535993693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:37.180253 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.180204    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:28:37.180381 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.180307    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:39.180269722 +0000 UTC m=+48.536082625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : configmap "kube-rbac-proxy" not found
Sep 10 14:28:37.399014 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.398922    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx"]
Sep 10 14:28:37.403067 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:37.403041    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc46da6_b07e_4f27_b980_1acad1ba60b0.slice/crio-47245eeceec4ad358ca21eb0d4a25208ed4bb287a062b7a19f8f9c4b1e868470 WatchSource:0}: Error finding container 47245eeceec4ad358ca21eb0d4a25208ed4bb287a062b7a19f8f9c4b1e868470: Status 404 returned error can't find the container with id 47245eeceec4ad358ca21eb0d4a25208ed4bb287a062b7a19f8f9c4b1e868470
Sep 10 14:28:37.640917 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.640884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx" event={"ID":"1bc46da6-b07e-4f27-b980-1acad1ba60b0","Type":"ContainerStarted","Data":"47245eeceec4ad358ca21eb0d4a25208ed4bb287a062b7a19f8f9c4b1e868470"}
Sep 10 14:28:37.784960 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.784925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:37.784960 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:37.784960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:37.785405 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.785098    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:45.785082662 +0000 UTC m=+55.140895552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : configmap references non-existent config key: service-ca.crt
Sep 10 14:28:37.785405 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.785140    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 14:28:37.785405 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:37.785199    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:45.785184169 +0000 UTC m=+55.140997054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : secret "router-metrics-certs-default" not found
Sep 10 14:28:38.289563 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.289533    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pksmv_6cddba93-d2fc-4e91-be06-a8559e276d53/node-ca/0.log"
Sep 10 14:28:38.648703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.648619    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8dk65"]
Sep 10 14:28:38.667729 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.667700    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8dk65"]
Sep 10 14:28:38.667892 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.667792    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.670113 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.670088    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 14:28:38.670239 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.670115    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-nsxhh\""
Sep 10 14:28:38.670679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.670621    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 14:28:38.670824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.670687    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 14:28:38.670824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.670704    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 14:28:38.685483 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.685458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:38.687615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.687601    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6sxkb_e4e1db6f-fa90-47e4-86eb-23713483bc05/serve-healthcheck-canary/0.log"
Sep 10 14:28:38.692888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.692868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z7c6\" (UniqueName: \"kubernetes.io/projected/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-kube-api-access-5z7c6\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.692969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.692912    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-signing-key\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.693012 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.692995    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-signing-cabundle\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.741324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.740911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:38.793661 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.793640    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-signing-key\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.793995 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.793689    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-signing-cabundle\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.793995 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.793948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z7c6\" (UniqueName: \"kubernetes.io/projected/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-kube-api-access-5z7c6\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.794340 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.794323    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-signing-cabundle\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.796179 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.796153    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-signing-key\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.801518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.801496    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z7c6\" (UniqueName: \"kubernetes.io/projected/08f3e0a2-a67e-4963-9623-d8ee5b4572a9-kube-api-access-5z7c6\") pod \"service-ca-75d99c484-8dk65\" (UID: \"08f3e0a2-a67e-4963-9623-d8ee5b4572a9\") " pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:38.980023 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:38.979989    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-8dk65"
Sep 10 14:28:39.196420 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.196399    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:39.196505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.196451    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:39.196553 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:39.196519    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:28:39.196588 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:39.196554    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:39.196588 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:39.196578    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:43.196561729 +0000 UTC m=+52.552374636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : configmap "kube-rbac-proxy" not found
Sep 10 14:28:39.196669 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:39.196607    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:43.196590793 +0000 UTC m=+52.552403682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:39.403747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.403720    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8dk65"]
Sep 10 14:28:39.406239 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:39.406217    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f3e0a2_a67e_4963_9623_d8ee5b4572a9.slice/crio-f370f589a0a7ecba9d57e239980537c4d447eb908c1695b4f19057e1e0e799fa WatchSource:0}: Error finding container f370f589a0a7ecba9d57e239980537c4d447eb908c1695b4f19057e1e0e799fa: Status 404 returned error can't find the container with id f370f589a0a7ecba9d57e239980537c4d447eb908c1695b4f19057e1e0e799fa
Sep 10 14:28:39.630672 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.630590    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:39.647410 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.647378    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-8dk65" event={"ID":"08f3e0a2-a67e-4963-9623-d8ee5b4572a9","Type":"ContainerStarted","Data":"f370f589a0a7ecba9d57e239980537c4d447eb908c1695b4f19057e1e0e799fa"}
Sep 10 14:28:39.649085 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.649063    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx" event={"ID":"1bc46da6-b07e-4f27-b980-1acad1ba60b0","Type":"ContainerStarted","Data":"1c25a5717190807ffb7bd1243da486566d62256ce158a8dd3acefb572c558e07"}
Sep 10 14:28:39.649199 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.649104    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx" event={"ID":"1bc46da6-b07e-4f27-b980-1acad1ba60b0","Type":"ContainerStarted","Data":"3992da364203914b59440e10a52d9a3aac9ca9e43d7e4a7fe7fe7599cecc5de6"}
Sep 10 14:28:39.667815 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:39.667769    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-6dpbx" podStartSLOduration=1.932210015 podStartE2EDuration="3.667754397s" totalImagesPullingTime="1.735544382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:39.666897858 +0000 UTC m=+49.022710785" watchObservedRunningTime="2026-09-10 14:28:39.667754397 +0000 UTC m=+49.023567304"
Sep 10 14:28:40.138025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:40.137989    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:40.406083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:40.405995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:40.406083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:40.406063    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:40.406309 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:40.406166    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:28:40.406309 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:40.406203    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:28:40.406309 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:40.406216    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7554c96755-ztg9w: secret "image-registry-tls" not found
Sep 10 14:28:40.406309 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:40.406248    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls podName:c7c25814-e802-4f67-95cd-12a0a7fcd941 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:56.406227391 +0000 UTC m=+65.762040293 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls") pod "dns-default-78xpw" (UID: "c7c25814-e802-4f67-95cd-12a0a7fcd941") : secret "dns-default-metrics-tls" not found
Sep 10 14:28:40.406309 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:40.406266    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls podName:4febf65c-8232-42e4-8e9e-89dfb9caa7db nodeName:}" failed. No retries permitted until 2026-09-10 14:28:56.406257352 +0000 UTC m=+65.762070238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls") pod "image-registry-7554c96755-ztg9w" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db") : secret "image-registry-tls" not found
Sep 10 14:28:42.659357 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:42.659317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-8dk65" event={"ID":"08f3e0a2-a67e-4963-9623-d8ee5b4572a9","Type":"ContainerStarted","Data":"14800ae8f91396cdc2cc9662e950a2076b184ddbc74e72d0d4bf3e3df04a2e97"}
Sep 10 14:28:42.675058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:42.675008    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-8dk65" podStartSLOduration=2.41115491 podStartE2EDuration="4.67499358s" totalImagesPullingTime="2.26383867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:42.673828359 +0000 UTC m=+52.029641267" watchObservedRunningTime="2026-09-10 14:28:42.67499358 +0000 UTC m=+52.030806486"
Sep 10 14:28:42.691955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:42.691929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:43.170490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.170454    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:43.231435 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.231350    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:43.231961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.231939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:43.232028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.232003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:43.232244 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:43.232230    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:43.232322 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:43.232308    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls podName:8df946e7-5aad-467f-a2bb-0e19d6f8ee50 nodeName:}" failed. No retries permitted until 2026-09-10 14:28:51.232270789 +0000 UTC m=+60.588083683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6j84z" (UID: "8df946e7-5aad-467f-a2bb-0e19d6f8ee50") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:28:43.383742 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.383708    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:43.484399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.484366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:43.591163 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.591131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:43.790152 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.790068    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:43.882915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:43.882885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:44.800716 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:44.800682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:44.901052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:44.901021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:45.256263 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:45.256232    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:45.464346 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:45.464318    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:45.664522 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:45.664448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:45.858140 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:45.858112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:45.858140 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:45.858142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:28:45.858649 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:45.858323    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:29:01.858298906 +0000 UTC m=+71.214111827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : configmap references non-existent config key: service-ca.crt
Sep 10 14:28:45.858649 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:45.858325    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 14:28:45.858649 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:28:45.858368    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs podName:8343c732-0f02-49d0-b506-c04b91e1e990 nodeName:}" failed. No retries permitted until 2026-09-10 14:29:01.858358048 +0000 UTC m=+71.214170937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs") pod "router-default-758b49888b-qhk2k" (UID: "8343c732-0f02-49d0-b506-c04b91e1e990") : secret "router-metrics-certs-default" not found
Sep 10 14:28:45.858649 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:45.858452    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:46.164381 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:46.164350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:46.654513 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:46.654481    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:46.855574 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:46.855518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:47.099121 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:47.099093    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:47.259548 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:47.259515    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:47.456665 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:47.456636    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:47.861296 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:47.861214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:47.961312 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:47.961271    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:48.065219 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:48.065189    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:48.459593 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:48.459561    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:49.262927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:49.262887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:49.441646 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:49.441618    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:49.656006 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:49.655926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:49.854403 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:49.854375    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:50.059209 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:50.059177    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:50.458909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:50.458876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:50.575194 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:50.575171    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8xhq"
Sep 10 14:28:50.687077 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:50.687040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:50.787836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:50.787766    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:51.059862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.059763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:51.305122 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.305085    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:51.307299 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.307266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8df946e7-5aad-467f-a2bb-0e19d6f8ee50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6j84z\" (UID: \"8df946e7-5aad-467f-a2bb-0e19d6f8ee50\") " pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:51.459533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.459500    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:51.461158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.461141    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fc42h\""
Sep 10 14:28:51.470017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.469999    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6j84z"
Sep 10 14:28:51.738329 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.738227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6j84z"]
Sep 10 14:28:51.742111 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:51.742086    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df946e7_5aad_467f_a2bb_0e19d6f8ee50.slice/crio-dd3f7d0a9456b7a4f7c1cb4e19b1576580c6b44e2144ef93a431731d3fb2f33d WatchSource:0}: Error finding container dd3f7d0a9456b7a4f7c1cb4e19b1576580c6b44e2144ef93a431731d3fb2f33d: Status 404 returned error can't find the container with id dd3f7d0a9456b7a4f7c1cb4e19b1576580c6b44e2144ef93a431731d3fb2f33d
Sep 10 14:28:51.857432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:51.857397    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:52.056484 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:52.056402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:52.458683 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:52.458652    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:52.690017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:52.689984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6j84z" event={"ID":"8df946e7-5aad-467f-a2bb-0e19d6f8ee50","Type":"ContainerStarted","Data":"6f246dacc67399518362252e4999118cc0a63e81129d12c511c69125c93efcd6"}
Sep 10 14:28:52.690165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:52.690069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6j84z" event={"ID":"8df946e7-5aad-467f-a2bb-0e19d6f8ee50","Type":"ContainerStarted","Data":"54718551794be969396450c8d4910fe07687f359201a8f2b92a2376b085d1007"}
Sep 10 14:28:52.690165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:52.690091    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6j84z" event={"ID":"8df946e7-5aad-467f-a2bb-0e19d6f8ee50","Type":"ContainerStarted","Data":"dd3f7d0a9456b7a4f7c1cb4e19b1576580c6b44e2144ef93a431731d3fb2f33d"}
Sep 10 14:28:53.061154 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:53.061066    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:53.458960 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:53.458929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:53.653403 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:53.653371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:53.858978 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:53.858943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:53.971560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:53.971529    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:54.072989 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.072909    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:54.127782 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.127752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:54.281906 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.281877    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:54.654088 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.654054    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:54.697422 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.697386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6j84z" event={"ID":"8df946e7-5aad-467f-a2bb-0e19d6f8ee50","Type":"ContainerStarted","Data":"3999dcc896510d161525a4f01593a1ed4353f44ee3778a852d4b2642aba85c53"}
Sep 10 14:28:54.714311 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.714246    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6j84z" podStartSLOduration=17.826089832 podStartE2EDuration="19.71423145s" totalImagesPullingTime="1.888141618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:28:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:54.713064753 +0000 UTC m=+64.068877690" watchObservedRunningTime="2026-09-10 14:28:54.71423145 +0000 UTC m=+64.070044356"
Sep 10 14:28:54.721526 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:54.721498    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.052293 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.052239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.105936 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.105903    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.458754 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.458726    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.563739 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.563709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.666571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.666544    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.726845 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.726771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.874347 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.874321    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:55.948240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.948205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:55.948375 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.948318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:55.950379 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.950362    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:28:55.950819 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.950805    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:28:55.960896 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.960874    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807de0e5-36be-4155-b273-fc7eb43f2f53-original-pull-secret\") pod \"global-pull-secret-syncer-ndntw\" (UID: \"807de0e5-36be-4155-b273-fc7eb43f2f53\") " pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:55.961016 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.960924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0a7d218-c488-478d-ad91-3e119f318ad0-metrics-certs\") pod \"network-metrics-daemon-57lnd\" (UID: \"d0a7d218-c488-478d-ad91-3e119f318ad0\") " pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:55.992463 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:55.992390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:56.028612 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.028577    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndntw"
Sep 10 14:28:56.034863 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.034844    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bf25v\""
Sep 10 14:28:56.043510 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.043488    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-57lnd"
Sep 10 14:28:56.049697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.049671    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:56.052222 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.052198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rps8h\" (UniqueName: \"kubernetes.io/projected/4ba171d4-5505-46f1-a8b7-95ce8474fcc0-kube-api-access-rps8h\") pod \"network-check-target-c6v7j\" (UID: \"4ba171d4-5505-46f1-a8b7-95ce8474fcc0\") " pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:56.325123 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.325053    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pptck\""
Sep 10 14:28:56.331912 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.331887    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ndntw"]
Sep 10 14:28:56.332888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.332869    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:56.334751 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:56.334730    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807de0e5_36be_4155_b273_fc7eb43f2f53.slice/crio-267ee00c4d0af5a67ed76b4abfe59df3a08228793fd62da0eb4a7b8988010390 WatchSource:0}: Error finding container 267ee00c4d0af5a67ed76b4abfe59df3a08228793fd62da0eb4a7b8988010390: Status 404 returned error can't find the container with id 267ee00c4d0af5a67ed76b4abfe59df3a08228793fd62da0eb4a7b8988010390
Sep 10 14:28:56.357956 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.357926    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-57lnd"]
Sep 10 14:28:56.358885 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:56.358861    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a7d218_c488_478d_ad91_3e119f318ad0.slice/crio-ab41bf3c85f41595b6f370eef821d6a93b5c674b99fde2933ce0d0a44bbefdb6 WatchSource:0}: Error finding container ab41bf3c85f41595b6f370eef821d6a93b5c674b99fde2933ce0d0a44bbefdb6: Status 404 returned error can't find the container with id ab41bf3c85f41595b6f370eef821d6a93b5c674b99fde2933ce0d0a44bbefdb6
Sep 10 14:28:56.454091 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.454047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:56.454312 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.454101    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:56.456477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.456449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7c25814-e802-4f67-95cd-12a0a7fcd941-metrics-tls\") pod \"dns-default-78xpw\" (UID: \"c7c25814-e802-4f67-95cd-12a0a7fcd941\") " pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:56.456574 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.456507    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"image-registry-7554c96755-ztg9w\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") " pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:56.600620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.600548    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6v7j"]
Sep 10 14:28:56.603451 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:56.603428    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba171d4_5505_46f1_a8b7_95ce8474fcc0.slice/crio-ea28157ed8080f13f8cc72e14ec521534685ed2677e7505dfe4afee5ef1b6fe6 WatchSource:0}: Error finding container ea28157ed8080f13f8cc72e14ec521534685ed2677e7505dfe4afee5ef1b6fe6: Status 404 returned error can't find the container with id ea28157ed8080f13f8cc72e14ec521534685ed2677e7505dfe4afee5ef1b6fe6
Sep 10 14:28:56.636754 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.636736    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2kwcf\""
Sep 10 14:28:56.645430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.645413    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:56.649369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.649351    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lsf8q\""
Sep 10 14:28:56.657727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.657708    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-78xpw"
Sep 10 14:28:56.711387 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.711267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-57lnd" event={"ID":"d0a7d218-c488-478d-ad91-3e119f318ad0","Type":"ContainerStarted","Data":"ab41bf3c85f41595b6f370eef821d6a93b5c674b99fde2933ce0d0a44bbefdb6"}
Sep 10 14:28:56.716048 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.716011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ndntw" event={"ID":"807de0e5-36be-4155-b273-fc7eb43f2f53","Type":"ContainerStarted","Data":"267ee00c4d0af5a67ed76b4abfe59df3a08228793fd62da0eb4a7b8988010390"}
Sep 10 14:28:56.719871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.719533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6v7j" event={"ID":"4ba171d4-5505-46f1-a8b7-95ce8474fcc0","Type":"ContainerStarted","Data":"d469a51dc95a579f1bd6be9380fe55810a56a7d49f71736a78a99c7c63f314aa"}
Sep 10 14:28:56.719871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.719575    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6v7j" event={"ID":"4ba171d4-5505-46f1-a8b7-95ce8474fcc0","Type":"ContainerStarted","Data":"ea28157ed8080f13f8cc72e14ec521534685ed2677e7505dfe4afee5ef1b6fe6"}
Sep 10 14:28:56.719871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.719655    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:28:56.735851 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.735797    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c6v7j" podStartSLOduration=65.735777288 podStartE2EDuration="1m5.735777288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:56.734642164 +0000 UTC m=+66.090455105" watchObservedRunningTime="2026-09-10 14:28:56.735777288 +0000 UTC m=+66.091590196"
Sep 10 14:28:56.919126 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.919051    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7554c96755-ztg9w"]
Sep 10 14:28:56.922175 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:56.922139    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4febf65c_8232_42e4_8e9e_89dfb9caa7db.slice/crio-31f44b9ddd87020612492476039743d06727d53f3c9f6e5cf392a6e9222a6bae WatchSource:0}: Error finding container 31f44b9ddd87020612492476039743d06727d53f3c9f6e5cf392a6e9222a6bae: Status 404 returned error can't find the container with id 31f44b9ddd87020612492476039743d06727d53f3c9f6e5cf392a6e9222a6bae
Sep 10 14:28:56.940167 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:56.940139    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-78xpw"]
Sep 10 14:28:56.945423 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:28:56.945394    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c25814_e802_4f67_95cd_12a0a7fcd941.slice/crio-876e5a8907829bc59eee40bded7a98bdad5db79988e61ef366e45ea513399662 WatchSource:0}: Error finding container 876e5a8907829bc59eee40bded7a98bdad5db79988e61ef366e45ea513399662: Status 404 returned error can't find the container with id 876e5a8907829bc59eee40bded7a98bdad5db79988e61ef366e45ea513399662
Sep 10 14:28:57.277477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.277431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:57.725651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.725609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-78xpw" event={"ID":"c7c25814-e802-4f67-95cd-12a0a7fcd941","Type":"ContainerStarted","Data":"876e5a8907829bc59eee40bded7a98bdad5db79988e61ef366e45ea513399662"}
Sep 10 14:28:57.727892 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.727806    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-57lnd" event={"ID":"d0a7d218-c488-478d-ad91-3e119f318ad0","Type":"ContainerStarted","Data":"3bb1a409ef36dbdb18a5add463615f9e0366815bea3fc31197a951d0cde53c03"}
Sep 10 14:28:57.727892 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.727859    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-57lnd" event={"ID":"d0a7d218-c488-478d-ad91-3e119f318ad0","Type":"ContainerStarted","Data":"e7bd0417da1946d462d80d8b076d04c0b404a1f4823a5b3755e4a0095fd6f835"}
Sep 10 14:28:57.729766 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.729738    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" event={"ID":"4febf65c-8232-42e4-8e9e-89dfb9caa7db","Type":"ContainerStarted","Data":"73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8"}
Sep 10 14:28:57.729876 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.729788    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" event={"ID":"4febf65c-8232-42e4-8e9e-89dfb9caa7db","Type":"ContainerStarted","Data":"31f44b9ddd87020612492476039743d06727d53f3c9f6e5cf392a6e9222a6bae"}
Sep 10 14:28:57.745439 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.745395    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-57lnd" podStartSLOduration=65.757590354 podStartE2EDuration="1m6.745381622s" totalImagesPullingTime="987.791268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:57.74416744 +0000 UTC m=+67.099980365" watchObservedRunningTime="2026-09-10 14:28:57.745381622 +0000 UTC m=+67.101194529"
Sep 10 14:28:57.764829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:57.764776    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" podStartSLOduration=66.764755653 podStartE2EDuration="1m6.764755653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:57.763470229 +0000 UTC m=+67.119283157" watchObservedRunningTime="2026-09-10 14:28:57.764755653 +0000 UTC m=+67.120568561"
Sep 10 14:28:58.735339 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:58.735306    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:28:59.740826 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:59.740787    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-78xpw" event={"ID":"c7c25814-e802-4f67-95cd-12a0a7fcd941","Type":"ContainerStarted","Data":"6ba3a9ec4b046a0f7d6e3c5fd2fa8fa47d1bc329191b58cbdc5b817e1298c50c"}
Sep 10 14:28:59.741294 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:59.740851    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-78xpw" event={"ID":"c7c25814-e802-4f67-95cd-12a0a7fcd941","Type":"ContainerStarted","Data":"8419475e1e2e0ad6eae40c6d9f4103400ae1a1748df609b67b720c8d01efccf2"}
Sep 10 14:28:59.758067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:28:59.757994    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-78xpw" podStartSLOduration=33.814123439 podStartE2EDuration="35.757976359s" totalImagesPullingTime="1.94385292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:59.757842208 +0000 UTC m=+69.113655140" watchObservedRunningTime="2026-09-10 14:28:59.757976359 +0000 UTC m=+69.113789268"
Sep 10 14:29:00.744839 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:00.744811    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-78xpw"
Sep 10 14:29:01.749523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.749487    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ndntw" event={"ID":"807de0e5-36be-4155-b273-fc7eb43f2f53","Type":"ContainerStarted","Data":"23aab73a023f3b09ed00c1d3896b8ed46b1d8b39d0ce064abc7b61caa876e852"}
Sep 10 14:29:01.765294 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.765231    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ndntw" podStartSLOduration=66.10709704 podStartE2EDuration="1m10.765215071s" totalImagesPullingTime="4.658118031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:01.763689742 +0000 UTC m=+71.119502661" watchObservedRunningTime="2026-09-10 14:29:01.765215071 +0000 UTC m=+71.121027977"
Sep 10 14:29:01.903218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.903177    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:01.903218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.903220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:01.903876 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.903852    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8343c732-0f02-49d0-b506-c04b91e1e990-service-ca-bundle\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:01.905558 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.905540    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8343c732-0f02-49d0-b506-c04b91e1e990-metrics-certs\") pod \"router-default-758b49888b-qhk2k\" (UID: \"8343c732-0f02-49d0-b506-c04b91e1e990\") " pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:01.949730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.949698    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"]
Sep 10 14:29:01.974430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.974387    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"]
Sep 10 14:29:01.974613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.974514    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:01.976863 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.976830    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 14:29:01.976972 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.976831    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 14:29:01.976972 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.976832    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 14:29:01.976972 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:01.976933    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 14:29:02.041419 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.041334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.104789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.104760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c0343452-297e-47b2-8bd3-628391069beb-klusterlet-config\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.104959 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.104796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0343452-297e-47b2-8bd3-628391069beb-tmp\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.104959 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.104814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn5nd\" (UniqueName: \"kubernetes.io/projected/c0343452-297e-47b2-8bd3-628391069beb-kube-api-access-zn5nd\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.105197 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.105180    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v8qsc\""
Sep 10 14:29:02.113200 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.113180    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:02.188951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.188924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.206338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.205492    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0343452-297e-47b2-8bd3-628391069beb-tmp\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.206338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.205537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn5nd\" (UniqueName: \"kubernetes.io/projected/c0343452-297e-47b2-8bd3-628391069beb-kube-api-access-zn5nd\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.206338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.205679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c0343452-297e-47b2-8bd3-628391069beb-klusterlet-config\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.206338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.205837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0343452-297e-47b2-8bd3-628391069beb-tmp\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.208593 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.208570    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c0343452-297e-47b2-8bd3-628391069beb-klusterlet-config\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.215515 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.215491    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn5nd\" (UniqueName: \"kubernetes.io/projected/c0343452-297e-47b2-8bd3-628391069beb-kube-api-access-zn5nd\") pod \"klusterlet-addon-workmgr-f47fc4769-sxkjc\" (UID: \"c0343452-297e-47b2-8bd3-628391069beb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.279469 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.279435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.284250 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.284232    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:02.378593 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.378559    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.409934 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.409899    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-758b49888b-qhk2k"]
Sep 10 14:29:02.413187 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:29:02.413150    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8343c732_0f02_49d0_b506_c04b91e1e990.slice/crio-a4cc4077a50c5869d7fe9a58c5ccc65f7ce1c2346176b52bbd5a204aa5988c93 WatchSource:0}: Error finding container a4cc4077a50c5869d7fe9a58c5ccc65f7ce1c2346176b52bbd5a204aa5988c93: Status 404 returned error can't find the container with id a4cc4077a50c5869d7fe9a58c5ccc65f7ce1c2346176b52bbd5a204aa5988c93
Sep 10 14:29:02.477969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.477938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.569832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.569761    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"]
Sep 10 14:29:02.573139 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:29:02.573113    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0343452_297e_47b2_8bd3_628391069beb.slice/crio-0f0ecdf3f78b74dc67b0901fd05412be76306c3ce914417c21ddb52910de2869 WatchSource:0}: Error finding container 0f0ecdf3f78b74dc67b0901fd05412be76306c3ce914417c21ddb52910de2869: Status 404 returned error can't find the container with id 0f0ecdf3f78b74dc67b0901fd05412be76306c3ce914417c21ddb52910de2869
Sep 10 14:29:02.754243 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.754203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-758b49888b-qhk2k" event={"ID":"8343c732-0f02-49d0-b506-c04b91e1e990","Type":"ContainerStarted","Data":"21066bd0787e7b7a9efffe86d033933438db33e425601f8386e424002fc03c26"}
Sep 10 14:29:02.754728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.754262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-758b49888b-qhk2k" event={"ID":"8343c732-0f02-49d0-b506-c04b91e1e990","Type":"ContainerStarted","Data":"a4cc4077a50c5869d7fe9a58c5ccc65f7ce1c2346176b52bbd5a204aa5988c93"}
Sep 10 14:29:02.755527 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.755506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc" event={"ID":"c0343452-297e-47b2-8bd3-628391069beb","Type":"ContainerStarted","Data":"0f0ecdf3f78b74dc67b0901fd05412be76306c3ce914417c21ddb52910de2869"}
Sep 10 14:29:02.777774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:02.777721    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-758b49888b-qhk2k" podStartSLOduration=33.777702168 podStartE2EDuration="33.777702168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:02.776652368 +0000 UTC m=+72.132465300" watchObservedRunningTime="2026-09-10 14:29:02.777702168 +0000 UTC m=+72.133515076"
Sep 10 14:29:03.009016 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.008977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:03.114095 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.114057    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:03.116720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.116696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:03.404474 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.404391    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:03.461364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.461320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:03.678890 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.678805    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:03.737966 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.737928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:03.760436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.760403    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:03.762146 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:03.762124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-758b49888b-qhk2k"
Sep 10 14:29:04.287251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:04.287183    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:04.355136 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:04.355074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:04.600156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:04.600053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:04.700511 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:04.700402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:05.700497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:05.700459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:05.760456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:05.760423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:05.883678 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:05.883642    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:05.942312 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:05.942267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:06.773215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:06.773185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc" event={"ID":"c0343452-297e-47b2-8bd3-628391069beb","Type":"ContainerStarted","Data":"b50361b8477f4041d3c0aea7a1c1e2833363ffb83d11a5e6974783d111a2c9c6"}
Sep 10 14:29:06.773717 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:06.773497    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:06.775137 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:06.775118    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc"
Sep 10 14:29:06.788737 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:06.788697    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f47fc4769-sxkjc" podStartSLOduration=2.127143683 podStartE2EDuration="5.788683735s" totalImagesPullingTime="3.661540052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:06.787718248 +0000 UTC m=+76.143531155" watchObservedRunningTime="2026-09-10 14:29:06.788683735 +0000 UTC m=+76.144496641"
Sep 10 14:29:07.389600 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:07.389564    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:07.489507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:07.489471    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:07.550743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:07.550704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:09.522470 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:09.522433    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:09.596729 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:09.596697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:09.692540 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:09.692510    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:09.877668 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:09.877584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:09.978984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:09.978950    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:10.035109 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:10.035076    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:10.751549 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:10.751515    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-78xpw"
Sep 10 14:29:11.891954 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:11.891923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:11.967011 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:11.966963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:12.877883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:12.877852    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:12.932797 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:12.932766    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:16.480050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:16.479942    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:16.543060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:16.543029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:16.649828 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:16.649795    2560 patch_prober.go:28] interesting pod/image-registry-7554c96755-ztg9w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 14:29:16.650010 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:16.649854    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" podUID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:29:19.745639 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:19.745609    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:29:19.873609 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:19.873571    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:20.003385 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:20.003308    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:20.063714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:20.063679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:20.658582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:20.658543    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:20.759601 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:20.759570    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:21.728762 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:21.728727    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:21.815361 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:21.815335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:21.900724 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:21.900696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:22.283647 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:22.283613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:22.476103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:22.476069    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:22.539557 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:22.539488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:25.307484 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:25.307454    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7554c96755-ztg9w"]
Sep 10 14:29:25.385376 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:25.385337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:25.484928 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:25.484895    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:25.973193 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:25.973161    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.045057 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:26.045023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:27.732528 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:27.732495    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c6v7j"
Sep 10 14:29:28.081255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:28.081173    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:28.606898 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:28.606869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:28.707747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:28.707718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:28.768156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:28.768135    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:28.870326 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:28.870241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:28.971784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:28.971755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:29.073115 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:29.073084    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:29.197684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:29.197654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:29.265352 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:29.265329    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:29.867057 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:29.867024    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.085096 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.085061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.124546 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.124469    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-48llk"]
Sep 10 14:29:30.129906 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.129883    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.131904 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.131880    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 14:29:30.132149 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.132135    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 14:29:30.132858 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.132836    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 14:29:30.132978 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.132873    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 14:29:30.132978 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.132846    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fq68n\""
Sep 10 14:29:30.133130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.133104    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 14:29:30.133306 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.133273    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 14:29:30.138393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.137921    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wsj79"]
Sep 10 14:29:30.141406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.141387    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.143911 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.143556    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 14:29:30.143911 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.143579    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vw687\""
Sep 10 14:29:30.143911 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.143762    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 14:29:30.143911 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.143806    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 14:29:30.144335 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.144317    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-48llk"]
Sep 10 14:29:30.182166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.182142    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.255731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.255698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.310071 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-tls\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310078    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-sys\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34b9b867-9bf7-4932-a6aa-085cb5e700ec-metrics-client-ca\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-textfile\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0e62375-71d1-498b-b78e-f52dd722895c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.310406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310332    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-wtmp\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310413    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.310524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.310524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310470    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e62375-71d1-498b-b78e-f52dd722895c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.310615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-root\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310555    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k58j\" (UniqueName: \"kubernetes.io/projected/34b9b867-9bf7-4932-a6aa-085cb5e700ec-kube-api-access-9k58j\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.310615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.310615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.310605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxcd\" (UniqueName: \"kubernetes.io/projected/e0e62375-71d1-498b-b78e-f52dd722895c-kube-api-access-4pxcd\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.362170 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.362141    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.411342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.411342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e62375-71d1-498b-b78e-f52dd722895c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411392    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-root\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k58j\" (UniqueName: \"kubernetes.io/projected/34b9b867-9bf7-4932-a6aa-085cb5e700ec-kube-api-access-9k58j\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxcd\" (UniqueName: \"kubernetes.io/projected/e0e62375-71d1-498b-b78e-f52dd722895c-kube-api-access-4pxcd\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-tls\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-sys\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34b9b867-9bf7-4932-a6aa-085cb5e700ec-metrics-client-ca\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-textfile\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0e62375-71d1-498b-b78e-f52dd722895c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.411969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411687    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-wtmp\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411845    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-wtmp\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.411969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.411907    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.412256 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:29:30.412012    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 14:29:30.412256 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:29:30.412067    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-tls podName:34b9b867-9bf7-4932-a6aa-085cb5e700ec nodeName:}" failed. No retries permitted until 2026-09-10 14:29:30.912047703 +0000 UTC m=+100.267860605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-tls") pod "node-exporter-wsj79" (UID: "34b9b867-9bf7-4932-a6aa-085cb5e700ec") : secret "node-exporter-tls" not found
Sep 10 14:29:30.412256 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.412065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.412447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.412331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-textfile\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.412447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.412393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-sys\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.412907 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.412884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e62375-71d1-498b-b78e-f52dd722895c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.413046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.412953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34b9b867-9bf7-4932-a6aa-085cb5e700ec-metrics-client-ca\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.413220 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.413198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0e62375-71d1-498b-b78e-f52dd722895c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.413305 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.413272    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34b9b867-9bf7-4932-a6aa-085cb5e700ec-root\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.415297 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.415255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.415514 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.415493    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0e62375-71d1-498b-b78e-f52dd722895c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.416212 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.416191    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.423576 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.423553    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxcd\" (UniqueName: \"kubernetes.io/projected/e0e62375-71d1-498b-b78e-f52dd722895c-kube-api-access-4pxcd\") pod \"kube-state-metrics-74cb57df7-48llk\" (UID: \"e0e62375-71d1-498b-b78e-f52dd722895c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.423961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.423943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k58j\" (UniqueName: \"kubernetes.io/projected/34b9b867-9bf7-4932-a6aa-085cb5e700ec-kube-api-access-9k58j\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.440567 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.440547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk"
Sep 10 14:29:30.457583 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.457557    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.600032 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.600000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.700640 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.700609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.719822 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.719793    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-48llk"]
Sep 10 14:29:30.723396 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:29:30.723367    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e62375_71d1_498b_b78e_f52dd722895c.slice/crio-782b489d4381f9bb634b32f43ad6740c2577c2763aa462a929c790c5a3db5d65 WatchSource:0}: Error finding container 782b489d4381f9bb634b32f43ad6740c2577c2763aa462a929c790c5a3db5d65: Status 404 returned error can't find the container with id 782b489d4381f9bb634b32f43ad6740c2577c2763aa462a929c790c5a3db5d65
Sep 10 14:29:30.857235 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.857197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk" event={"ID":"e0e62375-71d1-498b-b78e-f52dd722895c","Type":"ContainerStarted","Data":"782b489d4381f9bb634b32f43ad6740c2577c2763aa462a929c790c5a3db5d65"}
Sep 10 14:29:30.916511 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.916477    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-tls\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:30.917416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.917391    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:30.918923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:30.918901    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34b9b867-9bf7-4932-a6aa-085cb5e700ec-node-exporter-tls\") pod \"node-exporter-wsj79\" (UID: \"34b9b867-9bf7-4932-a6aa-085cb5e700ec\") " pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:31.051784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:31.051705    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wsj79"
Sep 10 14:29:31.063586 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:29:31.063559    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b9b867_9bf7_4932_a6aa_085cb5e700ec.slice/crio-28c53da845faff1509d7ee531f07c0345319d708891fa8432b42f76811300d5c WatchSource:0}: Error finding container 28c53da845faff1509d7ee531f07c0345319d708891fa8432b42f76811300d5c: Status 404 returned error can't find the container with id 28c53da845faff1509d7ee531f07c0345319d708891fa8432b42f76811300d5c
Sep 10 14:29:31.098937 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:31.098887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:31.184536 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:31.184503    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:31.299862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:31.299832    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:31.581359 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:31.581323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:31.862585 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:31.862506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsj79" event={"ID":"34b9b867-9bf7-4932-a6aa-085cb5e700ec","Type":"ContainerStarted","Data":"28c53da845faff1509d7ee531f07c0345319d708891fa8432b42f76811300d5c"}
Sep 10 14:29:32.867934 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:32.867900    2560 generic.go:356] "Generic (PLEG): container finished" podID="34b9b867-9bf7-4932-a6aa-085cb5e700ec" containerID="5bd31909fcb898bf836802d3fe2165a5aec731ca90c16dfbc60344dd6afa223b" exitCode=0
Sep 10 14:29:32.868388 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:32.867987    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsj79" event={"ID":"34b9b867-9bf7-4932-a6aa-085cb5e700ec","Type":"ContainerDied","Data":"5bd31909fcb898bf836802d3fe2165a5aec731ca90c16dfbc60344dd6afa223b"}
Sep 10 14:29:32.870356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:32.870270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk" event={"ID":"e0e62375-71d1-498b-b78e-f52dd722895c","Type":"ContainerStarted","Data":"789c0d14db537064e86dc36bb95a0515a0e46dca2590ae787a1e83b9221785ad"}
Sep 10 14:29:32.870356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:32.870338    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk" event={"ID":"e0e62375-71d1-498b-b78e-f52dd722895c","Type":"ContainerStarted","Data":"9bb6edaebe1250fcb84f1acc6a920b6b559cbcb7f537af1a0b22db4dfe0d069d"}
Sep 10 14:29:32.870356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:32.870362    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk" event={"ID":"e0e62375-71d1-498b-b78e-f52dd722895c","Type":"ContainerStarted","Data":"12c7874ddb5cff70240539fa046a67ce1559c50f34932ea6be2ae14b6bdaaa47"}
Sep 10 14:29:32.906186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:32.906144    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-48llk" podStartSLOduration=1.260787666 podStartE2EDuration="2.906129955s" totalImagesPullingTime="1.645342289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:32.904520423 +0000 UTC m=+102.260333330" watchObservedRunningTime="2026-09-10 14:29:32.906129955 +0000 UTC m=+102.261942901"
Sep 10 14:29:33.061154 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.061126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:33.161490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.161418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:33.234764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.234733    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:33.316770 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.316742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:33.509150 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.509120    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:33.875408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.875326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsj79" event={"ID":"34b9b867-9bf7-4932-a6aa-085cb5e700ec","Type":"ContainerStarted","Data":"24280e1ea04451b6069c8340dc5d3bb3946648695720f3f3e6806b28bba3d799"}
Sep 10 14:29:33.875747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.875410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsj79" event={"ID":"34b9b867-9bf7-4932-a6aa-085cb5e700ec","Type":"ContainerStarted","Data":"35ac5b026a91007c1ab48032c62a2f11e0ac996dca527a247c9b4eb0e34d43a4"}
Sep 10 14:29:33.897953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:33.897909    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wsj79" podStartSLOduration=2.518132891 podStartE2EDuration="3.89789431s" totalImagesPullingTime="1.303042996s" totalInitContainerRuntime="76.718423ms" isStatefulPod=true podCreationTimestamp="2026-09-10 14:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:33.896454496 +0000 UTC m=+103.252267418" watchObservedRunningTime="2026-09-10 14:29:33.89789431 +0000 UTC m=+103.253707217"
Sep 10 14:29:34.904075 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:34.904046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:35.003721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:35.003678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:35.106878 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:35.106849    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:35.218765 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:35.218735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:35.672429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:35.672358    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:35.769792 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:35.769752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:36.382231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:36.382198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:40.285128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:40.285094    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:40.387450 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:40.387424    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.034324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:42.034265    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.095426 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:42.095399    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.639862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:42.639829    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.741232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:42.741200    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:50.327239 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.327177    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" podUID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" containerName="registry" containerID="cri-o://73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8" gracePeriod=30
Sep 10 14:29:50.559014 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.558990    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:29:50.657433 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657366    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-bound-sa-token\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657433 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657400    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-certificates\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657437    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvnj2\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-kube-api-access-wvnj2\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657468    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657486    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-image-registry-private-configuration\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657503    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4febf65c-8232-42e4-8e9e-89dfb9caa7db-ca-trust-extracted\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657523    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-trusted-ca\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.657622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657562    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-installation-pull-secrets\") pod \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\" (UID: \"4febf65c-8232-42e4-8e9e-89dfb9caa7db\") "
Sep 10 14:29:50.658036 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.657973    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:50.658293 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.658249    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:50.660026 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.659979    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:50.660159 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.660034    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:50.660159 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.660070    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:50.660248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.660129    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:50.660248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.660204    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-kube-api-access-wvnj2" (OuterVolumeSpecName: "kube-api-access-wvnj2") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "kube-api-access-wvnj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:50.666064 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.666041    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4febf65c-8232-42e4-8e9e-89dfb9caa7db-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4febf65c-8232-42e4-8e9e-89dfb9caa7db" (UID: "4febf65c-8232-42e4-8e9e-89dfb9caa7db"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:50.670226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.670207    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:50.725498 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.725476    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:50.758668 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758644    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvnj2\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-kube-api-access-wvnj2\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758669    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758681    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-image-registry-private-configuration\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758691    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4febf65c-8232-42e4-8e9e-89dfb9caa7db-ca-trust-extracted\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758700    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-trusted-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758708    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4febf65c-8232-42e4-8e9e-89dfb9caa7db-installation-pull-secrets\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758716    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4febf65c-8232-42e4-8e9e-89dfb9caa7db-bound-sa-token\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.758731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.758724    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4febf65c-8232-42e4-8e9e-89dfb9caa7db-registry-certificates\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:29:50.936678 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.936651    2560 generic.go:356] "Generic (PLEG): container finished" podID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" containerID="73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8" exitCode=0
Sep 10 14:29:50.936880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.936693    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7554c96755-ztg9w"
Sep 10 14:29:50.936880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.936711    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" event={"ID":"4febf65c-8232-42e4-8e9e-89dfb9caa7db","Type":"ContainerDied","Data":"73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8"}
Sep 10 14:29:50.936880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.936754    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7554c96755-ztg9w" event={"ID":"4febf65c-8232-42e4-8e9e-89dfb9caa7db","Type":"ContainerDied","Data":"31f44b9ddd87020612492476039743d06727d53f3c9f6e5cf392a6e9222a6bae"}
Sep 10 14:29:50.936880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.936773    2560 scope.go:117] "RemoveContainer" containerID="73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8"
Sep 10 14:29:50.945363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.945343    2560 scope.go:117] "RemoveContainer" containerID="73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8"
Sep 10 14:29:50.945634 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:29:50.945614    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8\": container with ID starting with 73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8 not found: ID does not exist" containerID="73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8"
Sep 10 14:29:50.945714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.945647    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8"} err="failed to get container status \"73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8\": rpc error: code = NotFound desc = could not find container \"73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8\": container with ID starting with 73f072a33ec1b12b516805bcf10e8b0a48722bac0b5e8eb5896fc4456757bfe8 not found: ID does not exist"
Sep 10 14:29:50.956883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.956855    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7554c96755-ztg9w"]
Sep 10 14:29:50.960250 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:50.960230    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7554c96755-ztg9w"]
Sep 10 14:29:51.321437 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:51.321405    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" path="/var/lib/kubelet/pods/4febf65c-8232-42e4-8e9e-89dfb9caa7db/volumes"
Sep 10 14:29:52.741485 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:52.741459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:52.841672 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:52.841644    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:52.941810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:29:52.941778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:03.729642 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:03.729606    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:03.829404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:03.829370    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:05.988332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:05.988273    2560 generic.go:356] "Generic (PLEG): container finished" podID="1d2a0a89-de95-498a-b00e-7ad8ecc0e906" containerID="c92ca36505e0fe7a35cd9db761b368d369a85eb023b7529539aab3ef36928f42" exitCode=0
Sep 10 14:30:05.988332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:05.988324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n" event={"ID":"1d2a0a89-de95-498a-b00e-7ad8ecc0e906","Type":"ContainerDied","Data":"c92ca36505e0fe7a35cd9db761b368d369a85eb023b7529539aab3ef36928f42"}
Sep 10 14:30:05.988751 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:05.988677    2560 scope.go:117] "RemoveContainer" containerID="c92ca36505e0fe7a35cd9db761b368d369a85eb023b7529539aab3ef36928f42"
Sep 10 14:30:06.993952 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:06.993920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5sv6n" event={"ID":"1d2a0a89-de95-498a-b00e-7ad8ecc0e906","Type":"ContainerStarted","Data":"e37185249abc0ba4c1a9e4900be8aaa5a08fdd6a70ff6ff3b8308058ee02fb18"}
Sep 10 14:30:35.497347 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:35.497319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:42.342769 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:42.342739    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:46.243718 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:46.243643    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:46.315092 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:46.315054    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:54.116351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:54.116319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:54.220234 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:54.220200    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:55.077442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:55.077402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:58.113729 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:58.113693    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:58.215659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:30:58.215628    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.155093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:05.155064    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.256443 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:05.256410    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.321778 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:05.321750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.683780 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:05.683748    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.782811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:05.782781    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:36.928705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:36.928667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:36.981888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:36.981847    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.469600 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:49.469560    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.569532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:49.569503    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:50.068551 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.068513    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"]
Sep 10 14:31:50.068797 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.068785    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" containerName="registry"
Sep 10 14:31:50.068842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.068798    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" containerName="registry"
Sep 10 14:31:50.068882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.068853    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4febf65c-8232-42e4-8e9e-89dfb9caa7db" containerName="registry"
Sep 10 14:31:50.071850 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.071830    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.073893 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.073870    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:31:50.074018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.073911    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:31:50.074599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.074573    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nvm5\""
Sep 10 14:31:50.079945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.079920    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"]
Sep 10 14:31:50.116497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.116462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9srf6\" (UniqueName: \"kubernetes.io/projected/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-kube-api-access-9srf6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.116497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.116496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.116699 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.116544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.217128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.217092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9srf6\" (UniqueName: \"kubernetes.io/projected/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-kube-api-access-9srf6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.217128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.217129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.217360 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.217321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.217564 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.217544    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.217660 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.217639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.224898 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.224875    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9srf6\" (UniqueName: \"kubernetes.io/projected/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-kube-api-access-9srf6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.381063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.380976    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:31:50.649429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:50.649384    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"]
Sep 10 14:31:50.651643 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:31:50.651615    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc6f8f6d_28d2_4ba5_b7f3_15064b66bc6a.slice/crio-5418c76400b659bda0fd0b7099bc171a151adbbdbb3128622cb9fff2d2b71b48 WatchSource:0}: Error finding container 5418c76400b659bda0fd0b7099bc171a151adbbdbb3128622cb9fff2d2b71b48: Status 404 returned error can't find the container with id 5418c76400b659bda0fd0b7099bc171a151adbbdbb3128622cb9fff2d2b71b48
Sep 10 14:31:51.351232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:51.351189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f" event={"ID":"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a","Type":"ContainerStarted","Data":"5418c76400b659bda0fd0b7099bc171a151adbbdbb3128622cb9fff2d2b71b48"}
Sep 10 14:31:56.369927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:56.369887    2560 generic.go:356] "Generic (PLEG): container finished" podID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerID="9f468f911882bdbed16d8f393ecdc03af536091ea62afbdc78f3f504e6c28f08" exitCode=0
Sep 10 14:31:56.370410 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:56.369973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f" event={"ID":"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a","Type":"ContainerDied","Data":"9f468f911882bdbed16d8f393ecdc03af536091ea62afbdc78f3f504e6c28f08"}
Sep 10 14:31:58.379073 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:58.379045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f" event={"ID":"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a","Type":"ContainerStarted","Data":"d91c5a5c81123266022a5d8178c9b7c98b5be1dfef9db64c985856bdf1deee39"}
Sep 10 14:31:59.383967 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:59.383937    2560 generic.go:356] "Generic (PLEG): container finished" podID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerID="d91c5a5c81123266022a5d8178c9b7c98b5be1dfef9db64c985856bdf1deee39" exitCode=0
Sep 10 14:31:59.384385 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:31:59.384010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f" event={"ID":"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a","Type":"ContainerDied","Data":"d91c5a5c81123266022a5d8178c9b7c98b5be1dfef9db64c985856bdf1deee39"}
Sep 10 14:32:05.409408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:05.409326    2560 generic.go:356] "Generic (PLEG): container finished" podID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerID="26452a51845dfe524a39b70830f3c148a94b24a9e60b197d15967080e433d249" exitCode=0
Sep 10 14:32:05.409408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:05.409374    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f" event={"ID":"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a","Type":"ContainerDied","Data":"26452a51845dfe524a39b70830f3c148a94b24a9e60b197d15967080e433d249"}
Sep 10 14:32:06.531743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.531719    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:32:06.650819 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.650784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-util\") pod \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") "
Sep 10 14:32:06.650990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.650835    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9srf6\" (UniqueName: \"kubernetes.io/projected/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-kube-api-access-9srf6\") pod \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") "
Sep 10 14:32:06.650990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.650858    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-bundle\") pod \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\" (UID: \"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a\") "
Sep 10 14:32:06.651710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.651681    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-bundle" (OuterVolumeSpecName: "bundle") pod "dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" (UID: "dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:06.652965 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.652940    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-kube-api-access-9srf6" (OuterVolumeSpecName: "kube-api-access-9srf6") pod "dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" (UID: "dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a"). InnerVolumeSpecName "kube-api-access-9srf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:06.655482 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.655456    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-util" (OuterVolumeSpecName: "util") pod "dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" (UID: "dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:06.751808 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.751770    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:06.751808 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.751802    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9srf6\" (UniqueName: \"kubernetes.io/projected/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-kube-api-access-9srf6\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:06.751808 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:06.751811    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:07.418390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:07.418356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f" event={"ID":"dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a","Type":"ContainerDied","Data":"5418c76400b659bda0fd0b7099bc171a151adbbdbb3128622cb9fff2d2b71b48"}
Sep 10 14:32:07.418390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:07.418379    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mht5f"
Sep 10 14:32:07.418599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:07.418403    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5418c76400b659bda0fd0b7099bc171a151adbbdbb3128622cb9fff2d2b71b48"
Sep 10 14:32:11.089728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:11.089695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:11.884494 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:11.884460    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.112576 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112529    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"]
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112780    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="util"
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112792    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="util"
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112803    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="extract"
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112808    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="extract"
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112822    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="pull"
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112827    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="pull"
Sep 10 14:32:13.113030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.112868    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc6f8f6d-28d2-4ba5-b7f3-15064b66bc6a" containerName="extract"
Sep 10 14:32:13.122669 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.122648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.126507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.126481    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qnz97\""
Sep 10 14:32:13.126719 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.126703    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 14:32:13.126770 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.126732    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:32:13.133299 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.133255    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"]
Sep 10 14:32:13.198327 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.198300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13be97ad-4510-4b72-ae66-c513b3244aff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7dhtp\" (UID: \"13be97ad-4510-4b72-ae66-c513b3244aff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.198472 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.198336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smsg2\" (UniqueName: \"kubernetes.io/projected/13be97ad-4510-4b72-ae66-c513b3244aff-kube-api-access-smsg2\") pod \"cert-manager-operator-controller-manager-55c444bf86-7dhtp\" (UID: \"13be97ad-4510-4b72-ae66-c513b3244aff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.299351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.299250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13be97ad-4510-4b72-ae66-c513b3244aff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7dhtp\" (UID: \"13be97ad-4510-4b72-ae66-c513b3244aff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.299351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.299312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smsg2\" (UniqueName: \"kubernetes.io/projected/13be97ad-4510-4b72-ae66-c513b3244aff-kube-api-access-smsg2\") pod \"cert-manager-operator-controller-manager-55c444bf86-7dhtp\" (UID: \"13be97ad-4510-4b72-ae66-c513b3244aff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.299653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.299634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13be97ad-4510-4b72-ae66-c513b3244aff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7dhtp\" (UID: \"13be97ad-4510-4b72-ae66-c513b3244aff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.309814 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.309783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smsg2\" (UniqueName: \"kubernetes.io/projected/13be97ad-4510-4b72-ae66-c513b3244aff-kube-api-access-smsg2\") pod \"cert-manager-operator-controller-manager-55c444bf86-7dhtp\" (UID: \"13be97ad-4510-4b72-ae66-c513b3244aff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.430794 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.430703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"
Sep 10 14:32:13.726063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:13.725970    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp"]
Sep 10 14:32:13.729135 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:13.729099    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13be97ad_4510_4b72_ae66_c513b3244aff.slice/crio-f109d0b45861638c4fea2f4228948f19956b92b3abd4a1a448a873b9f897574f WatchSource:0}: Error finding container f109d0b45861638c4fea2f4228948f19956b92b3abd4a1a448a873b9f897574f: Status 404 returned error can't find the container with id f109d0b45861638c4fea2f4228948f19956b92b3abd4a1a448a873b9f897574f
Sep 10 14:32:14.448809 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:14.448772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp" event={"ID":"13be97ad-4510-4b72-ae66-c513b3244aff","Type":"ContainerStarted","Data":"f109d0b45861638c4fea2f4228948f19956b92b3abd4a1a448a873b9f897574f"}
Sep 10 14:32:14.885848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:14.885749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:14.986390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:14.986351    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:16.459025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:16.458983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp" event={"ID":"13be97ad-4510-4b72-ae66-c513b3244aff","Type":"ContainerStarted","Data":"5d9dcc563e97833dd426e10ff9227022e4733a03648d730ada56a6845dec6753"}
Sep 10 14:32:16.503482 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:16.503424    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7dhtp" podStartSLOduration=1.208267536 podStartE2EDuration="3.503406364s" totalImagesPullingTime="2.295138828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:16.502796399 +0000 UTC m=+265.858609306" watchObservedRunningTime="2026-09-10 14:32:16.503406364 +0000 UTC m=+265.859219271"
Sep 10 14:32:16.897782 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:16.897701    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.111151 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.111118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.289572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.289543    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"]
Sep 10 14:32:17.312844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.312812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.316955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.316931    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:32:17.317105 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.317045    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nvm5\""
Sep 10 14:32:17.317692 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.317669    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:32:17.323102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.323080    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"]
Sep 10 14:32:17.432606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.432566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drn9k\" (UniqueName: \"kubernetes.io/projected/7a29b26f-dabb-409a-a5df-f254e164317d-kube-api-access-drn9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.432606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.432607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.432809 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.432636    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.436069 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.436042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.512590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.512559    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h8bnq"]
Sep 10 14:32:17.529266 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.529239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.533171 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.533146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.533313 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.533207    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.533313 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.533304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drn9k\" (UniqueName: \"kubernetes.io/projected/7a29b26f-dabb-409a-a5df-f254e164317d-kube-api-access-drn9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.533572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.533553    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.533658 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.533598    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.534553 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.534534    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 14:32:17.534771 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.534749    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 14:32:17.536811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.536789    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sz746\""
Sep 10 14:32:17.538598 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.538579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h8bnq"]
Sep 10 14:32:17.552564 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.552510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drn9k\" (UniqueName: \"kubernetes.io/projected/7a29b26f-dabb-409a-a5df-f254e164317d-kube-api-access-drn9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.624183 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.624151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:17.634503 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.634475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn5hj\" (UniqueName: \"kubernetes.io/projected/cb3afc58-ae4e-431f-831f-650b4d5ad562-kube-api-access-vn5hj\") pod \"cert-manager-webhook-6486ffd997-h8bnq\" (UID: \"cb3afc58-ae4e-431f-831f-650b4d5ad562\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.634616 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.634528    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3afc58-ae4e-431f-831f-650b4d5ad562-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h8bnq\" (UID: \"cb3afc58-ae4e-431f-831f-650b4d5ad562\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.735571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.735537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn5hj\" (UniqueName: \"kubernetes.io/projected/cb3afc58-ae4e-431f-831f-650b4d5ad562-kube-api-access-vn5hj\") pod \"cert-manager-webhook-6486ffd997-h8bnq\" (UID: \"cb3afc58-ae4e-431f-831f-650b4d5ad562\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.735716 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.735608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3afc58-ae4e-431f-831f-650b4d5ad562-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h8bnq\" (UID: \"cb3afc58-ae4e-431f-831f-650b4d5ad562\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.755049 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.755023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3afc58-ae4e-431f-831f-650b4d5ad562-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h8bnq\" (UID: \"cb3afc58-ae4e-431f-831f-650b4d5ad562\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.755853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.755834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn5hj\" (UniqueName: \"kubernetes.io/projected/cb3afc58-ae4e-431f-831f-650b4d5ad562-kube-api-access-vn5hj\") pod \"cert-manager-webhook-6486ffd997-h8bnq\" (UID: \"cb3afc58-ae4e-431f-831f-650b4d5ad562\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.847988 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.847910    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:17.927396 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:17.927203    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"]
Sep 10 14:32:17.929922 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:17.929891    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a29b26f_dabb_409a_a5df_f254e164317d.slice/crio-8df55c19258b54c9450ab8306ba9672d51fcef9ebd2f5782fb0de4cb13ab6945 WatchSource:0}: Error finding container 8df55c19258b54c9450ab8306ba9672d51fcef9ebd2f5782fb0de4cb13ab6945: Status 404 returned error can't find the container with id 8df55c19258b54c9450ab8306ba9672d51fcef9ebd2f5782fb0de4cb13ab6945
Sep 10 14:32:18.145108 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:18.145030    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h8bnq"]
Sep 10 14:32:18.148485 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:18.148457    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb3afc58_ae4e_431f_831f_650b4d5ad562.slice/crio-fa2a6db6b443259f4b27d30074032dff72c2ad3d4b4755e8b6aecaccc103acf7 WatchSource:0}: Error finding container fa2a6db6b443259f4b27d30074032dff72c2ad3d4b4755e8b6aecaccc103acf7: Status 404 returned error can't find the container with id fa2a6db6b443259f4b27d30074032dff72c2ad3d4b4755e8b6aecaccc103acf7
Sep 10 14:32:18.467764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:18.467728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq" event={"ID":"cb3afc58-ae4e-431f-831f-650b4d5ad562","Type":"ContainerStarted","Data":"fa2a6db6b443259f4b27d30074032dff72c2ad3d4b4755e8b6aecaccc103acf7"}
Sep 10 14:32:18.469266 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:18.469244    2560 generic.go:356] "Generic (PLEG): container finished" podID="7a29b26f-dabb-409a-a5df-f254e164317d" containerID="c4c8e1db4b5f34a8b749095501c05bda67342be3c8aaa4a311aa730844c26f53" exitCode=0
Sep 10 14:32:18.469395 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:18.469316    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc" event={"ID":"7a29b26f-dabb-409a-a5df-f254e164317d","Type":"ContainerDied","Data":"c4c8e1db4b5f34a8b749095501c05bda67342be3c8aaa4a311aa730844c26f53"}
Sep 10 14:32:18.469395 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:18.469380    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc" event={"ID":"7a29b26f-dabb-409a-a5df-f254e164317d","Type":"ContainerStarted","Data":"8df55c19258b54c9450ab8306ba9672d51fcef9ebd2f5782fb0de4cb13ab6945"}
Sep 10 14:32:18.972563 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:18.972536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:19.744190 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:19.744154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:21.484637 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:21.484535    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:21.486653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:21.486625    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq" event={"ID":"cb3afc58-ae4e-431f-831f-650b4d5ad562","Type":"ContainerStarted","Data":"b873373f5ec70f168cebfb8dc4d629bceaf812107994acd4ab3187f41e0a9789"}
Sep 10 14:32:21.486786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:21.486751    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:21.505077 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:21.505026    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq" podStartSLOduration=1.747822024 podStartE2EDuration="4.50500876s" totalImagesPullingTime="2.757186736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:21.50416166 +0000 UTC m=+270.859974568" watchObservedRunningTime="2026-09-10 14:32:21.50500876 +0000 UTC m=+270.860821667"
Sep 10 14:32:21.585488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:21.585453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:22.492829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:22.492793    2560 generic.go:356] "Generic (PLEG): container finished" podID="7a29b26f-dabb-409a-a5df-f254e164317d" containerID="d8c14aba97bc09798e701b4edce421f796761b451a05d3db1eb7dce337324b01" exitCode=0
Sep 10 14:32:22.493246 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:22.492865    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc" event={"ID":"7a29b26f-dabb-409a-a5df-f254e164317d","Type":"ContainerDied","Data":"d8c14aba97bc09798e701b4edce421f796761b451a05d3db1eb7dce337324b01"}
Sep 10 14:32:23.500011 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:23.499954    2560 generic.go:356] "Generic (PLEG): container finished" podID="7a29b26f-dabb-409a-a5df-f254e164317d" containerID="c9957286c7731073a23e0b236113854e12cfa03c4d3f05807e0ade9ccc14fa7b" exitCode=0
Sep 10 14:32:23.500482 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:23.500018    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc" event={"ID":"7a29b26f-dabb-409a-a5df-f254e164317d","Type":"ContainerDied","Data":"c9957286c7731073a23e0b236113854e12cfa03c4d3f05807e0ade9ccc14fa7b"}
Sep 10 14:32:24.114723 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.114691    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"]
Sep 10 14:32:24.119289 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.119259    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.121599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.121575    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2bmq6\""
Sep 10 14:32:24.126099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.126082    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"]
Sep 10 14:32:24.187056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.187021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c98a28-7934-438a-894f-9a23d8d90470-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-76g5t\" (UID: \"a2c98a28-7934-438a-894f-9a23d8d90470\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.187217 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.187074    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbphl\" (UniqueName: \"kubernetes.io/projected/a2c98a28-7934-438a-894f-9a23d8d90470-kube-api-access-zbphl\") pod \"cert-manager-cainjector-559cdd8c87-76g5t\" (UID: \"a2c98a28-7934-438a-894f-9a23d8d90470\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.287725 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.287684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbphl\" (UniqueName: \"kubernetes.io/projected/a2c98a28-7934-438a-894f-9a23d8d90470-kube-api-access-zbphl\") pod \"cert-manager-cainjector-559cdd8c87-76g5t\" (UID: \"a2c98a28-7934-438a-894f-9a23d8d90470\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.287903 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.287775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c98a28-7934-438a-894f-9a23d8d90470-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-76g5t\" (UID: \"a2c98a28-7934-438a-894f-9a23d8d90470\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.297828 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.297793    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c98a28-7934-438a-894f-9a23d8d90470-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-76g5t\" (UID: \"a2c98a28-7934-438a-894f-9a23d8d90470\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.302425 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.302394    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbphl\" (UniqueName: \"kubernetes.io/projected/a2c98a28-7934-438a-894f-9a23d8d90470-kube-api-access-zbphl\") pod \"cert-manager-cainjector-559cdd8c87-76g5t\" (UID: \"a2c98a28-7934-438a-894f-9a23d8d90470\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.428547 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.428436    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"
Sep 10 14:32:24.619979 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.619954    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:24.690577 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.690551    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drn9k\" (UniqueName: \"kubernetes.io/projected/7a29b26f-dabb-409a-a5df-f254e164317d-kube-api-access-drn9k\") pod \"7a29b26f-dabb-409a-a5df-f254e164317d\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") "
Sep 10 14:32:24.690714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.690642    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-util\") pod \"7a29b26f-dabb-409a-a5df-f254e164317d\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") "
Sep 10 14:32:24.690714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.690689    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-bundle\") pod \"7a29b26f-dabb-409a-a5df-f254e164317d\" (UID: \"7a29b26f-dabb-409a-a5df-f254e164317d\") "
Sep 10 14:32:24.691078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.691055    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-bundle" (OuterVolumeSpecName: "bundle") pod "7a29b26f-dabb-409a-a5df-f254e164317d" (UID: "7a29b26f-dabb-409a-a5df-f254e164317d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:24.692642 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.692619    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a29b26f-dabb-409a-a5df-f254e164317d-kube-api-access-drn9k" (OuterVolumeSpecName: "kube-api-access-drn9k") pod "7a29b26f-dabb-409a-a5df-f254e164317d" (UID: "7a29b26f-dabb-409a-a5df-f254e164317d"). InnerVolumeSpecName "kube-api-access-drn9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:24.695124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.695094    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-util" (OuterVolumeSpecName: "util") pod "7a29b26f-dabb-409a-a5df-f254e164317d" (UID: "7a29b26f-dabb-409a-a5df-f254e164317d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:24.706463 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.706375    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-76g5t"]
Sep 10 14:32:24.708998 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:24.708977    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2c98a28_7934_438a_894f_9a23d8d90470.slice/crio-55345e74683551165cd00df24a11db57c1f7c47c207fbaf2ea309c80e0627541 WatchSource:0}: Error finding container 55345e74683551165cd00df24a11db57c1f7c47c207fbaf2ea309c80e0627541: Status 404 returned error can't find the container with id 55345e74683551165cd00df24a11db57c1f7c47c207fbaf2ea309c80e0627541
Sep 10 14:32:24.791348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.791318    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:24.791348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.791345    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drn9k\" (UniqueName: \"kubernetes.io/projected/7a29b26f-dabb-409a-a5df-f254e164317d-kube-api-access-drn9k\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:24.791491 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:24.791355    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a29b26f-dabb-409a-a5df-f254e164317d-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:25.510244 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.510207    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc" event={"ID":"7a29b26f-dabb-409a-a5df-f254e164317d","Type":"ContainerDied","Data":"8df55c19258b54c9450ab8306ba9672d51fcef9ebd2f5782fb0de4cb13ab6945"}
Sep 10 14:32:25.510472 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.510259    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8df55c19258b54c9450ab8306ba9672d51fcef9ebd2f5782fb0de4cb13ab6945"
Sep 10 14:32:25.510472 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.510220    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7nxfc"
Sep 10 14:32:25.511953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.511930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t" event={"ID":"a2c98a28-7934-438a-894f-9a23d8d90470","Type":"ContainerStarted","Data":"6722ed146e4ddbe90ffe3309f34a29c23caa937ac18cf2bae5dad82034ecdf5d"}
Sep 10 14:32:25.512077 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.511965    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t" event={"ID":"a2c98a28-7934-438a-894f-9a23d8d90470","Type":"ContainerStarted","Data":"55345e74683551165cd00df24a11db57c1f7c47c207fbaf2ea309c80e0627541"}
Sep 10 14:32:25.527179 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.527155    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:25.627890 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.627810    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:25.661118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:25.661065    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-76g5t" podStartSLOduration=1.6610487250000001 podStartE2EDuration="1.661048725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:25.542660423 +0000 UTC m=+274.898473332" watchObservedRunningTime="2026-09-10 14:32:25.661048725 +0000 UTC m=+275.016861632"
Sep 10 14:32:27.495356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:27.495326    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h8bnq"
Sep 10 14:32:27.554628 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:27.554598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:28.997211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997132    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-77ghj"]
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997441    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="pull"
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997454    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="pull"
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997472    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="util"
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997477    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="util"
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997487    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="extract"
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997493    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="extract"
Sep 10 14:32:28.997590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:28.997533    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a29b26f-dabb-409a-a5df-f254e164317d" containerName="extract"
Sep 10 14:32:29.000906 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.000888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.003059 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.003034    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bv58q\""
Sep 10 14:32:29.010051 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.010028    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-77ghj"]
Sep 10 14:32:29.128611 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.128576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hzbm\" (UniqueName: \"kubernetes.io/projected/9d34d606-c6b2-46d9-98f6-29fc8d1bd962-kube-api-access-4hzbm\") pod \"cert-manager-55b4d954d5-77ghj\" (UID: \"9d34d606-c6b2-46d9-98f6-29fc8d1bd962\") " pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.128789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.128622    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d34d606-c6b2-46d9-98f6-29fc8d1bd962-bound-sa-token\") pod \"cert-manager-55b4d954d5-77ghj\" (UID: \"9d34d606-c6b2-46d9-98f6-29fc8d1bd962\") " pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.229550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.229510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d34d606-c6b2-46d9-98f6-29fc8d1bd962-bound-sa-token\") pod \"cert-manager-55b4d954d5-77ghj\" (UID: \"9d34d606-c6b2-46d9-98f6-29fc8d1bd962\") " pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.229700 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.229581    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hzbm\" (UniqueName: \"kubernetes.io/projected/9d34d606-c6b2-46d9-98f6-29fc8d1bd962-kube-api-access-4hzbm\") pod \"cert-manager-55b4d954d5-77ghj\" (UID: \"9d34d606-c6b2-46d9-98f6-29fc8d1bd962\") " pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.238742 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.238706    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hzbm\" (UniqueName: \"kubernetes.io/projected/9d34d606-c6b2-46d9-98f6-29fc8d1bd962-kube-api-access-4hzbm\") pod \"cert-manager-55b4d954d5-77ghj\" (UID: \"9d34d606-c6b2-46d9-98f6-29fc8d1bd962\") " pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.239205 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.239186    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d34d606-c6b2-46d9-98f6-29fc8d1bd962-bound-sa-token\") pod \"cert-manager-55b4d954d5-77ghj\" (UID: \"9d34d606-c6b2-46d9-98f6-29fc8d1bd962\") " pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.311050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.310954    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-77ghj"
Sep 10 14:32:29.586615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:29.586532    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-77ghj"]
Sep 10 14:32:29.589468 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:29.589437    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d34d606_c6b2_46d9_98f6_29fc8d1bd962.slice/crio-ae4200e1082c24132755ac18d183022123566586df050ea0ba5bbc99532dd6c6 WatchSource:0}: Error finding container ae4200e1082c24132755ac18d183022123566586df050ea0ba5bbc99532dd6c6: Status 404 returned error can't find the container with id ae4200e1082c24132755ac18d183022123566586df050ea0ba5bbc99532dd6c6
Sep 10 14:32:30.534702 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:30.534657    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-77ghj" event={"ID":"9d34d606-c6b2-46d9-98f6-29fc8d1bd962","Type":"ContainerStarted","Data":"3c6f92ed39153d98c8f464f88a8d5a8192cdaf9598f389320cc8c20da5e35843"}
Sep 10 14:32:30.534702 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:30.534703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-77ghj" event={"ID":"9d34d606-c6b2-46d9-98f6-29fc8d1bd962","Type":"ContainerStarted","Data":"ae4200e1082c24132755ac18d183022123566586df050ea0ba5bbc99532dd6c6"}
Sep 10 14:32:31.880217 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:31.880181    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:31.978854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:31.978821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:39.012106 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:39.012075    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:39.106507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:39.106475    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:40.766725 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.766693    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:40.885337 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.885266    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-77ghj" podStartSLOduration=12.885246696 podStartE2EDuration="12.885246696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:30.562007124 +0000 UTC m=+279.917820047" watchObservedRunningTime="2026-09-10 14:32:40.885246696 +0000 UTC m=+290.241059603"
Sep 10 14:32:40.886218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.886200    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"]
Sep 10 14:32:40.889597 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.889578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:40.891617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.891594    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:32:40.892642 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.892614    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nvm5\""
Sep 10 14:32:40.892748 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.892685    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:32:40.898037 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:40.898013    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"]
Sep 10 14:32:41.021515 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.021422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.021515 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.021487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.021711 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.021562    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwhv9\" (UniqueName: \"kubernetes.io/projected/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-kube-api-access-lwhv9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.121946 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.121906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.122117 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.121965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwhv9\" (UniqueName: \"kubernetes.io/projected/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-kube-api-access-lwhv9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.122117 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.122008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.122325 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.122303    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.122398 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.122376    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.129549 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.129520    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwhv9\" (UniqueName: \"kubernetes.io/projected/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-kube-api-access-lwhv9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.199189 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.199143    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:41.467319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.467234    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"]
Sep 10 14:32:41.470105 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:41.470075    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ddd3e0c_0832_4aca_97be_7cdbe64c9dbb.slice/crio-f2644d7bbc1f6dae2c35be2ef262589b58e1ad64c626bf7df0545844853eb8ef WatchSource:0}: Error finding container f2644d7bbc1f6dae2c35be2ef262589b58e1ad64c626bf7df0545844853eb8ef: Status 404 returned error can't find the container with id f2644d7bbc1f6dae2c35be2ef262589b58e1ad64c626bf7df0545844853eb8ef
Sep 10 14:32:41.581412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.581381    2560 generic.go:356] "Generic (PLEG): container finished" podID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerID="f455a23cad360e456e46a3acd009121b4371127d9c0ea7efecc70d88ef2549dc" exitCode=0
Sep 10 14:32:41.581560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.581470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b" event={"ID":"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb","Type":"ContainerDied","Data":"f455a23cad360e456e46a3acd009121b4371127d9c0ea7efecc70d88ef2549dc"}
Sep 10 14:32:41.581560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:41.581536    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b" event={"ID":"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb","Type":"ContainerStarted","Data":"f2644d7bbc1f6dae2c35be2ef262589b58e1ad64c626bf7df0545844853eb8ef"}
Sep 10 14:32:42.588355 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:42.588319    2560 generic.go:356] "Generic (PLEG): container finished" podID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerID="a62c2becb823e5e7964ea73365a8c8068ebeadf05e5c2fc93369cf5f0f0eb487" exitCode=0
Sep 10 14:32:42.588727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:42.588418    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b" event={"ID":"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb","Type":"ContainerDied","Data":"a62c2becb823e5e7964ea73365a8c8068ebeadf05e5c2fc93369cf5f0f0eb487"}
Sep 10 14:32:43.594782 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:43.594748    2560 generic.go:356] "Generic (PLEG): container finished" podID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerID="aefd8eb8c33f35cac67277d67643f2f0732c59c1313ddec69d34de9d79eeb31c" exitCode=0
Sep 10 14:32:43.595175 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:43.594810    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b" event={"ID":"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb","Type":"ContainerDied","Data":"aefd8eb8c33f35cac67277d67643f2f0732c59c1313ddec69d34de9d79eeb31c"}
Sep 10 14:32:44.725798 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.725774    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:44.851961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.851870    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-util\") pod \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") "
Sep 10 14:32:44.851961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.851916    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwhv9\" (UniqueName: \"kubernetes.io/projected/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-kube-api-access-lwhv9\") pod \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") "
Sep 10 14:32:44.851961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.851936    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-bundle\") pod \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\" (UID: \"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb\") "
Sep 10 14:32:44.852762 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.852731    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-bundle" (OuterVolumeSpecName: "bundle") pod "3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" (UID: "3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:44.854055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.854027    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-kube-api-access-lwhv9" (OuterVolumeSpecName: "kube-api-access-lwhv9") pod "3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" (UID: "3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb"). InnerVolumeSpecName "kube-api-access-lwhv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:44.857434 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.857409    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-util" (OuterVolumeSpecName: "util") pod "3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" (UID: "3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:44.952533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.952494    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:44.952533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.952529    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwhv9\" (UniqueName: \"kubernetes.io/projected/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-kube-api-access-lwhv9\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:44.952702 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:44.952540    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:32:45.604684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:45.604605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b" event={"ID":"3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb","Type":"ContainerDied","Data":"f2644d7bbc1f6dae2c35be2ef262589b58e1ad64c626bf7df0545844853eb8ef"}
Sep 10 14:32:45.604684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:45.604652    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af47g8b"
Sep 10 14:32:45.604854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:45.604687    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2644d7bbc1f6dae2c35be2ef262589b58e1ad64c626bf7df0545844853eb8ef"
Sep 10 14:32:49.524493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:49.524458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:49.583041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:49.583013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:51.231650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:51.231622    2560 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 14:32:54.683777 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.683741    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"]
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684013    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="extract"
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684024    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="extract"
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684035    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="util"
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684040    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="util"
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684054    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="pull"
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684060    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="pull"
Sep 10 14:32:54.688635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.684104    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ddd3e0c-0832-4aca-97be-7cdbe64c9dbb" containerName="extract"
Sep 10 14:32:54.688860 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.688849    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.691104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.691080    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:32:54.691775 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.691755    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nvm5\""
Sep 10 14:32:54.691775 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.691764    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:32:54.695119 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.695097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"]
Sep 10 14:32:54.723463 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.723427    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.723463 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.723461    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdj9s\" (UniqueName: \"kubernetes.io/projected/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-kube-api-access-kdj9s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.723653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.723490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.824712 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.824672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.824882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.824747    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.824882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.824771    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdj9s\" (UniqueName: \"kubernetes.io/projected/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-kube-api-access-kdj9s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.825120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.825097    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.825184 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.825108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.847526 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.847497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdj9s\" (UniqueName: \"kubernetes.io/projected/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-kube-api-access-kdj9s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:54.909559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.909529    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:54.998868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:54.998835    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:55.278022 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.277996    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"]
Sep 10 14:32:55.279975 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:55.279943    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6474f3_4ade_4e75_9f98_a3f5a6848f85.slice/crio-15b923731da08f6e32e28f18291d2be94c89e49d4f1ac96d0f0fda2f2db946d4 WatchSource:0}: Error finding container 15b923731da08f6e32e28f18291d2be94c89e49d4f1ac96d0f0fda2f2db946d4: Status 404 returned error can't find the container with id 15b923731da08f6e32e28f18291d2be94c89e49d4f1ac96d0f0fda2f2db946d4
Sep 10 14:32:55.649044 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.648959    2560 generic.go:356] "Generic (PLEG): container finished" podID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerID="58e7bb2aedfd84b3bf3c00d87ac43f31124c7646861975e4bd32ee0b5af6df61" exitCode=0
Sep 10 14:32:55.649181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.649038    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz" event={"ID":"1c6474f3-4ade-4e75-9f98-a3f5a6848f85","Type":"ContainerDied","Data":"58e7bb2aedfd84b3bf3c00d87ac43f31124c7646861975e4bd32ee0b5af6df61"}
Sep 10 14:32:55.649181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.649099    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz" event={"ID":"1c6474f3-4ade-4e75-9f98-a3f5a6848f85","Type":"ContainerStarted","Data":"15b923731da08f6e32e28f18291d2be94c89e49d4f1ac96d0f0fda2f2db946d4"}
Sep 10 14:32:55.649500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.649486    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:32:55.886097 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.886059    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"]
Sep 10 14:32:55.889431 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.889409    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:55.891889 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.891864    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 14:32:55.892040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.891864    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 14:32:55.892040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.891925    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 14:32:55.892202 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.892186    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 14:32:55.892304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.892272    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2nvw6\""
Sep 10 14:32:55.904169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.904117    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"]
Sep 10 14:32:55.932187 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.932150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b595p\" (UniqueName: \"kubernetes.io/projected/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-kube-api-access-b595p\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:55.932397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.932204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:55.932397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.932289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:55.932397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.932367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-tmp\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:55.932536 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:55.932416    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-manifests\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.032933 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.032895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b595p\" (UniqueName: \"kubernetes.io/projected/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-kube-api-access-b595p\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.033121 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.032942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.033121 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.033064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.033236 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.033145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-tmp\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.033236 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.033191    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-manifests\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.033578 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.033557    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-manifests\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.035372 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.035352    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-tmp\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.035483 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.035464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.035523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.035489    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.041568 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.041536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b595p\" (UniqueName: \"kubernetes.io/projected/cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1-kube-api-access-b595p\") pod \"opendatahub-operator-controller-manager-77dd699645-7lfz4\" (UID: \"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1\") " pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.199877 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.199839    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:32:56.494458 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.493969    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"]
Sep 10 14:32:56.497018 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:32:56.496979    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9c4207_0c89_48fb_bc01_4b8f3cf30bb1.slice/crio-15e1ecd1b2c119f0516f68a7d422bdaf44cf5f781c57bf1ba7ce2d7b3cbfd687 WatchSource:0}: Error finding container 15e1ecd1b2c119f0516f68a7d422bdaf44cf5f781c57bf1ba7ce2d7b3cbfd687: Status 404 returned error can't find the container with id 15e1ecd1b2c119f0516f68a7d422bdaf44cf5f781c57bf1ba7ce2d7b3cbfd687
Sep 10 14:32:56.654713 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.654669    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4" event={"ID":"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1","Type":"ContainerStarted","Data":"15e1ecd1b2c119f0516f68a7d422bdaf44cf5f781c57bf1ba7ce2d7b3cbfd687"}
Sep 10 14:32:56.656541 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.656514    2560 generic.go:356] "Generic (PLEG): container finished" podID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerID="1fc81e71b41b73776bf77d0cdafa485a0265726bf58accfe9beb98a122eb49c8" exitCode=0
Sep 10 14:32:56.656650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.656573    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz" event={"ID":"1c6474f3-4ade-4e75-9f98-a3f5a6848f85","Type":"ContainerDied","Data":"1fc81e71b41b73776bf77d0cdafa485a0265726bf58accfe9beb98a122eb49c8"}
Sep 10 14:32:56.939357 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:56.939331    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:57.669937 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:57.669899    2560 generic.go:356] "Generic (PLEG): container finished" podID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerID="51a6e75a70d9d154382f72ad2245345c5b8b64fab52f115b1fffcab26e8a0221" exitCode=0
Sep 10 14:32:57.670106 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:57.670025    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz" event={"ID":"1c6474f3-4ade-4e75-9f98-a3f5a6848f85","Type":"ContainerDied","Data":"51a6e75a70d9d154382f72ad2245345c5b8b64fab52f115b1fffcab26e8a0221"}
Sep 10 14:32:59.924118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.924091    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:32:59.967393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.967372    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-util\") pod \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") "
Sep 10 14:32:59.967552 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.967415    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdj9s\" (UniqueName: \"kubernetes.io/projected/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-kube-api-access-kdj9s\") pod \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") "
Sep 10 14:32:59.967552 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.967439    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-bundle\") pod \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\" (UID: \"1c6474f3-4ade-4e75-9f98-a3f5a6848f85\") "
Sep 10 14:32:59.968348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.968319    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-bundle" (OuterVolumeSpecName: "bundle") pod "1c6474f3-4ade-4e75-9f98-a3f5a6848f85" (UID: "1c6474f3-4ade-4e75-9f98-a3f5a6848f85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:59.969783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.969751    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-kube-api-access-kdj9s" (OuterVolumeSpecName: "kube-api-access-kdj9s") pod "1c6474f3-4ade-4e75-9f98-a3f5a6848f85" (UID: "1c6474f3-4ade-4e75-9f98-a3f5a6848f85"). InnerVolumeSpecName "kube-api-access-kdj9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:59.973940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:32:59.973916    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-util" (OuterVolumeSpecName: "util") pod "1c6474f3-4ade-4e75-9f98-a3f5a6848f85" (UID: "1c6474f3-4ade-4e75-9f98-a3f5a6848f85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:33:00.068973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.068943    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:00.068973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.068972    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdj9s\" (UniqueName: \"kubernetes.io/projected/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-kube-api-access-kdj9s\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:00.069160 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.068983    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c6474f3-4ade-4e75-9f98-a3f5a6848f85-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:00.686531 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.686497    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz"
Sep 10 14:33:00.686531 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.686518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lmpsz" event={"ID":"1c6474f3-4ade-4e75-9f98-a3f5a6848f85","Type":"ContainerDied","Data":"15b923731da08f6e32e28f18291d2be94c89e49d4f1ac96d0f0fda2f2db946d4"}
Sep 10 14:33:00.686746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.686573    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15b923731da08f6e32e28f18291d2be94c89e49d4f1ac96d0f0fda2f2db946d4"
Sep 10 14:33:00.688136 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:00.688109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4" event={"ID":"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1","Type":"ContainerStarted","Data":"eeaddafdc498c21a2068043c5164131400deb81a42067a92842f9a816b017d29"}
Sep 10 14:33:01.694495 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:01.694463    2560 generic.go:356] "Generic (PLEG): container finished" podID="cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1" containerID="eeaddafdc498c21a2068043c5164131400deb81a42067a92842f9a816b017d29" exitCode=0
Sep 10 14:33:01.694880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:01.694521    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4" event={"ID":"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1","Type":"ContainerDied","Data":"eeaddafdc498c21a2068043c5164131400deb81a42067a92842f9a816b017d29"}
Sep 10 14:33:02.703813 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:02.703774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4" event={"ID":"cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1","Type":"ContainerStarted","Data":"9f308e90d3a40a715a132fdc83fed23301733fee40fc42d52c0976042fe79740"}
Sep 10 14:33:02.704205 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:02.703869    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:33:02.726247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:02.726190    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4" podStartSLOduration=2.8832088110000003 podStartE2EDuration="7.726174742s" totalImagesPullingTime="3.788414908s" totalInitContainerRuntime="1.054551023s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:02.725163667 +0000 UTC m=+312.080976574" watchObservedRunningTime="2026-09-10 14:33:02.726174742 +0000 UTC m=+312.081987649"
Sep 10 14:33:03.345745 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:03.345692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:03.473841 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:03.473792    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:07.697416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:07.697380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:07.793741 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:07.793703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:13.710238 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:13.710205    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77dd699645-7lfz4"
Sep 10 14:33:15.377583 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.377547    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:15.545319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.545259    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:15.705649 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.705621    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:15.776628 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.776594    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"]
Sep 10 14:33:15.776946 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.776929    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="extract"
Sep 10 14:33:15.777025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.776949    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="extract"
Sep 10 14:33:15.777025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.776984    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="util"
Sep 10 14:33:15.777025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.776993    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="util"
Sep 10 14:33:15.777025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.777005    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="pull"
Sep 10 14:33:15.777025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.777015    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="pull"
Sep 10 14:33:15.777260 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.777085    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c6474f3-4ade-4e75-9f98-a3f5a6848f85" containerName="extract"
Sep 10 14:33:15.780045 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.780024    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.783369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.783337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 14:33:15.783369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.783359    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pxcmb\""
Sep 10 14:33:15.783722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.783701    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 14:33:15.795251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.795218    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"]
Sep 10 14:33:15.886112 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.886065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/754e6c11-7151-4277-b955-c56bfc53da50-tls-certs\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.886335 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.886125    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79vxl\" (UniqueName: \"kubernetes.io/projected/754e6c11-7151-4277-b955-c56bfc53da50-kube-api-access-79vxl\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.886335 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.886161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/754e6c11-7151-4277-b955-c56bfc53da50-tmp\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.956629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.956537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:15.987017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.986981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/754e6c11-7151-4277-b955-c56bfc53da50-tls-certs\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.987158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.987027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79vxl\" (UniqueName: \"kubernetes.io/projected/754e6c11-7151-4277-b955-c56bfc53da50-kube-api-access-79vxl\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.987158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.987064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/754e6c11-7151-4277-b955-c56bfc53da50-tmp\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.989251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.989224    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/754e6c11-7151-4277-b955-c56bfc53da50-tmp\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.989562 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.989541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/754e6c11-7151-4277-b955-c56bfc53da50-tls-certs\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:15.997935 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:15.997912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79vxl\" (UniqueName: \"kubernetes.io/projected/754e6c11-7151-4277-b955-c56bfc53da50-kube-api-access-79vxl\") pod \"kube-auth-proxy-6c68946dcf-spdl2\" (UID: \"754e6c11-7151-4277-b955-c56bfc53da50\") " pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:16.089345 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:16.089304    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"
Sep 10 14:33:16.358031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:16.357851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2"]
Sep 10 14:33:16.360692 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:33:16.360666    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod754e6c11_7151_4277_b955_c56bfc53da50.slice/crio-18f3eea4de92a6dcc7bd0decd3a24baacea803547dcb965fc43a8373efed04be WatchSource:0}: Error finding container 18f3eea4de92a6dcc7bd0decd3a24baacea803547dcb965fc43a8373efed04be: Status 404 returned error can't find the container with id 18f3eea4de92a6dcc7bd0decd3a24baacea803547dcb965fc43a8373efed04be
Sep 10 14:33:16.767533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:16.767495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2" event={"ID":"754e6c11-7151-4277-b955-c56bfc53da50","Type":"ContainerStarted","Data":"18f3eea4de92a6dcc7bd0decd3a24baacea803547dcb965fc43a8373efed04be"}
Sep 10 14:33:17.037600 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.037459    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"]
Sep 10 14:33:17.041005 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.040971    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.042743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.042716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:17.043612 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.043584    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:33:17.043765 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.043742    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 14:33:17.044165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.043904    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dh959\""
Sep 10 14:33:17.044165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.043949    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 14:33:17.044165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.044091    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 14:33:17.057993 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.057963    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"]
Sep 10 14:33:17.096344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcvg9\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-kube-api-access-rcvg9\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.096344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096135    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.096344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.096344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096193    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.096344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096238    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.096344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.096784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.096404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/855c39a1-13db-416d-b00c-6074a814c8ce-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.145148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.145095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:17.197874 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.197834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.198058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.197916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/855c39a1-13db-416d-b00c-6074a814c8ce-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.198058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.197970    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcvg9\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-kube-api-access-rcvg9\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.198058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.197997    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.198058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.198025    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.198058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.198050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.198341 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.198089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.199705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.199609    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.200987 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.200942    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.201400 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.201366    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.201606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.201582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/855c39a1-13db-416d-b00c-6074a814c8ce-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.203417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.203393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.206943 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.206921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcvg9\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-kube-api-access-rcvg9\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.207349 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.207318    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7fqc9\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.354061 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.353968    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:17.635508 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:17.635345    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"]
Sep 10 14:33:18.013212 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:18.013170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:18.993020 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:33:18.992977    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855c39a1_13db_416d_b00c_6074a814c8ce.slice/crio-11a3698adc017449427b9fdea628a79eea3de8f6f8619259087b6fe1e6df1ede WatchSource:0}: Error finding container 11a3698adc017449427b9fdea628a79eea3de8f6f8619259087b6fe1e6df1ede: Status 404 returned error can't find the container with id 11a3698adc017449427b9fdea628a79eea3de8f6f8619259087b6fe1e6df1ede
Sep 10 14:33:19.784180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:19.784138    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" event={"ID":"855c39a1-13db-416d-b00c-6074a814c8ce","Type":"ContainerStarted","Data":"11a3698adc017449427b9fdea628a79eea3de8f6f8619259087b6fe1e6df1ede"}
Sep 10 14:33:19.785626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:19.785596    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2" event={"ID":"754e6c11-7151-4277-b955-c56bfc53da50","Type":"ContainerStarted","Data":"c48a0e53ff92fd05c59c1836dfa4c736d8ae1b0d30bbf1b5091ee5a4efc8e0a1"}
Sep 10 14:33:19.800532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:19.800481    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c68946dcf-spdl2" podStartSLOduration=2.117054708 podStartE2EDuration="4.800466264s" totalImagesPullingTime="2.683411556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:19.799820397 +0000 UTC m=+329.155633317" watchObservedRunningTime="2026-09-10 14:33:19.800466264 +0000 UTC m=+329.156279170"
Sep 10 14:33:22.433168 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:22.433124    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 14:33:22.433510 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:22.433199    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 14:33:22.805394 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:22.805354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" event={"ID":"855c39a1-13db-416d-b00c-6074a814c8ce","Type":"ContainerStarted","Data":"4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356"}
Sep 10 14:33:22.805589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:22.805494    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:22.824730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:22.824675    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" podStartSLOduration=2.387356516 podStartE2EDuration="5.824657676s" totalImagesPullingTime="3.43730116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:22.821966171 +0000 UTC m=+332.177779076" watchObservedRunningTime="2026-09-10 14:33:22.824657676 +0000 UTC m=+332.180470580"
Sep 10 14:33:23.461767 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:23.461730    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:23.561786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:23.561755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:23.812289 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:23.812201    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-7fqc9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 14:33:23.812424 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:23.812314    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" podUID="855c39a1-13db-416d-b00c-6074a814c8ce" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:33:24.065971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:24.065893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:24.290731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:24.290697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:24.577191 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:24.577165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:26.418066 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:26.418029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:26.510970 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:26.510932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:26.610506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:26.610470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:26.812368 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:26.812336    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:26.895878 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:26.895850    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:26.990467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:26.990425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:27.089598 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:27.089525    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:35.757630 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.757597    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:35.908481 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.908440    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"]
Sep 10 14:33:35.916272 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.916242    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:35.919454 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.919355    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-l5pzg\""
Sep 10 14:33:35.919454 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.919431    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 14:33:35.925381 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.925353    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"]
Sep 10 14:33:35.957810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:35.957778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:36.062435 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.062347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2888bbca-5f69-43b4-88cf-e3168498706f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qblnw\" (UID: \"2888bbca-5f69-43b4-88cf-e3168498706f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.062435 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.062401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnk99\" (UniqueName: \"kubernetes.io/projected/2888bbca-5f69-43b4-88cf-e3168498706f-kube-api-access-wnk99\") pod \"llmisvc-controller-manager-777c6bd7d5-qblnw\" (UID: \"2888bbca-5f69-43b4-88cf-e3168498706f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.091558 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.091527    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:36.163236 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.163187    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2888bbca-5f69-43b4-88cf-e3168498706f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qblnw\" (UID: \"2888bbca-5f69-43b4-88cf-e3168498706f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.163236 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.163242    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnk99\" (UniqueName: \"kubernetes.io/projected/2888bbca-5f69-43b4-88cf-e3168498706f-kube-api-access-wnk99\") pod \"llmisvc-controller-manager-777c6bd7d5-qblnw\" (UID: \"2888bbca-5f69-43b4-88cf-e3168498706f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.165964 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.165941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2888bbca-5f69-43b4-88cf-e3168498706f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qblnw\" (UID: \"2888bbca-5f69-43b4-88cf-e3168498706f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.183561 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.183537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnk99\" (UniqueName: \"kubernetes.io/projected/2888bbca-5f69-43b4-88cf-e3168498706f-kube-api-access-wnk99\") pod \"llmisvc-controller-manager-777c6bd7d5-qblnw\" (UID: \"2888bbca-5f69-43b4-88cf-e3168498706f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.210941 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.210907    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:36.229923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.229886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:36.334371 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.334275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:36.501053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.501029    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"]
Sep 10 14:33:36.503360 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:33:36.503332    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2888bbca_5f69_43b4_88cf_e3168498706f.slice/crio-5bc0388cc997d322cd96cbdccbee6bc86ed8a2906ba623463fe6af6de413f347 WatchSource:0}: Error finding container 5bc0388cc997d322cd96cbdccbee6bc86ed8a2906ba623463fe6af6de413f347: Status 404 returned error can't find the container with id 5bc0388cc997d322cd96cbdccbee6bc86ed8a2906ba623463fe6af6de413f347
Sep 10 14:33:36.617269 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.617185    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"]
Sep 10 14:33:36.617442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.617418    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" podUID="855c39a1-13db-416d-b00c-6074a814c8ce" containerName="discovery" containerID="cri-o://4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356" gracePeriod=30
Sep 10 14:33:36.850494 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.850470    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:36.870050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.869974    2560 generic.go:356] "Generic (PLEG): container finished" podID="855c39a1-13db-416d-b00c-6074a814c8ce" containerID="4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356" exitCode=0
Sep 10 14:33:36.870050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.870012    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"
Sep 10 14:33:36.870221 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.870051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" event={"ID":"855c39a1-13db-416d-b00c-6074a814c8ce","Type":"ContainerDied","Data":"4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356"}
Sep 10 14:33:36.870221 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.870091    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9" event={"ID":"855c39a1-13db-416d-b00c-6074a814c8ce","Type":"ContainerDied","Data":"11a3698adc017449427b9fdea628a79eea3de8f6f8619259087b6fe1e6df1ede"}
Sep 10 14:33:36.870221 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.870116    2560 scope.go:117] "RemoveContainer" containerID="4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356"
Sep 10 14:33:36.871208 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.871180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw" event={"ID":"2888bbca-5f69-43b4-88cf-e3168498706f","Type":"ContainerStarted","Data":"5bc0388cc997d322cd96cbdccbee6bc86ed8a2906ba623463fe6af6de413f347"}
Sep 10 14:33:36.878568 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.878551    2560 scope.go:117] "RemoveContainer" containerID="4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356"
Sep 10 14:33:36.878828 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:33:36.878805    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356\": container with ID starting with 4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356 not found: ID does not exist" containerID="4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356"
Sep 10 14:33:36.878893 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.878841    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356"} err="failed to get container status \"4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356\": rpc error: code = NotFound desc = could not find container \"4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356\": container with ID starting with 4f63bccf906392d8a2ec62b2f3089dd742c635a09b5d1060b91af7aac56bf356 not found: ID does not exist"
Sep 10 14:33:36.969928 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.969891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-istio-token\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.969938    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/855c39a1-13db-416d-b00c-6074a814c8ce-local-certs\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.969968    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-cacerts\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.969998    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-ca-configmap\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.970022    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-kubeconfig\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.970045    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcvg9\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-kube-api-access-rcvg9\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.970092    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-dns-cert\") pod \"855c39a1-13db-416d-b00c-6074a814c8ce\" (UID: \"855c39a1-13db-416d-b00c-6074a814c8ce\") "
Sep 10 14:33:36.970480 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.970453    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:33:36.972499 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.972473    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:33:36.972628 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.972570    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-cacerts" (OuterVolumeSpecName: "cacerts") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:33:36.972725 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.972707    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-kube-api-access-rcvg9" (OuterVolumeSpecName: "kube-api-access-rcvg9") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "kube-api-access-rcvg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:33:36.972774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.972729    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:33:36.972809 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.972795    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:33:36.972881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:36.972864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/855c39a1-13db-416d-b00c-6074a814c8ce-local-certs" (OuterVolumeSpecName: "local-certs") pod "855c39a1-13db-416d-b00c-6074a814c8ce" (UID: "855c39a1-13db-416d-b00c-6074a814c8ce"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:33:37.070765 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070732    2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/855c39a1-13db-416d-b00c-6074a814c8ce-local-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.070765 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070762    2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-cacerts\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.070765 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070771    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-ca-configmap\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.070981 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070780    2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-kubeconfig\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.070981 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070788    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcvg9\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-kube-api-access-rcvg9\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.070981 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070797    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/855c39a1-13db-416d-b00c-6074a814c8ce-istio-csr-dns-cert\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.070981 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.070805    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855c39a1-13db-416d-b00c-6074a814c8ce-istio-token\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:33:37.195430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.195396    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"]
Sep 10 14:33:37.198651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.198626    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7fqc9"]
Sep 10 14:33:37.318879 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:37.318846    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855c39a1-13db-416d-b00c-6074a814c8ce" path="/var/lib/kubelet/pods/855c39a1-13db-416d-b00c-6074a814c8ce/volumes"
Sep 10 14:33:39.890193 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:39.890152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw" event={"ID":"2888bbca-5f69-43b4-88cf-e3168498706f","Type":"ContainerStarted","Data":"59076cc0ee1e3b11d8a9880d5bc7fc7c14b69f59194c5b81759de271515b38ae"}
Sep 10 14:33:39.890624 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:39.890309    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:33:39.906465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:33:39.906410    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw" podStartSLOduration=2.134170834 podStartE2EDuration="4.906393685s" totalImagesPullingTime="2.772222851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:39.905074094 +0000 UTC m=+349.260887025" watchObservedRunningTime="2026-09-10 14:33:39.906393685 +0000 UTC m=+349.262206592"
Sep 10 14:34:10.898085 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:10.898052    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qblnw"
Sep 10 14:34:13.073299 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:13.073240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:13.225447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:13.225414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:13.296368 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:13.296336    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:13.829947 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:13.829913    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:13.881469 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:13.881436    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:24.083333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:24.083299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:24.184522 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:24.184487    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:24.284680 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:24.284646    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:27.645709 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:27.645676    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:27.742610 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:27.742580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:31.318721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.318687    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"]
Sep 10 14:34:31.319074 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.318960    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855c39a1-13db-416d-b00c-6074a814c8ce" containerName="discovery"
Sep 10 14:34:31.319074 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.318971    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="855c39a1-13db-416d-b00c-6074a814c8ce" containerName="discovery"
Sep 10 14:34:31.319074 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.319024    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="855c39a1-13db-416d-b00c-6074a814c8ce" containerName="discovery"
Sep 10 14:34:31.322017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.322000    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.324485 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.324464    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:34:31.324769 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.324752    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:34:31.325373 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.325356    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nvm5\""
Sep 10 14:34:31.333566 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.333539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"]
Sep 10 14:34:31.386063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.386029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.386248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.386068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hh7q\" (UniqueName: \"kubernetes.io/projected/8ec22522-2394-42ff-93fb-ac540aeed741-kube-api-access-6hh7q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.386248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.386191    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.414392 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.414359    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"]
Sep 10 14:34:31.417588 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.417571    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.425932 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.425909    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"]
Sep 10 14:34:31.486591 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.486554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.486591 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.486595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.486857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.486618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.486857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.486642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hh7q\" (UniqueName: \"kubernetes.io/projected/8ec22522-2394-42ff-93fb-ac540aeed741-kube-api-access-6hh7q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.486857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.486667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbshl\" (UniqueName: \"kubernetes.io/projected/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-kube-api-access-nbshl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.486857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.486708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.487126 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.487101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.487181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.487130    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.496332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.496306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hh7q\" (UniqueName: \"kubernetes.io/projected/8ec22522-2394-42ff-93fb-ac540aeed741-kube-api-access-6hh7q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.511851 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.511821    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"]
Sep 10 14:34:31.515364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.515347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.523784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.523758    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"]
Sep 10 14:34:31.587556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587454    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.587556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.587784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.587784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.587784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbshl\" (UniqueName: \"kubernetes.io/projected/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-kube-api-access-nbshl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.587784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587673    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-967xs\" (UniqueName: \"kubernetes.io/projected/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-kube-api-access-967xs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.587994 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.588041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.587991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.595108 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.595078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbshl\" (UniqueName: \"kubernetes.io/projected/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-kube-api-access-nbshl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.619644 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.619612    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"]
Sep 10 14:34:31.623040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.623025    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.631629 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.631600    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:31.631747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.631691    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"]
Sep 10 14:34:31.688929 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.688899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.689068 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.688969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-967xs\" (UniqueName: \"kubernetes.io/projected/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-kube-api-access-967xs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.689068 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.689026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.689068 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.689058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.689228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.689093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.689228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.689145    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4vjj\" (UniqueName: \"kubernetes.io/projected/aae394bf-7dcf-4ecc-9013-2cb6130d6573-kube-api-access-s4vjj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.689449 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.689362    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.689707 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.689680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.697637 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.697607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-967xs\" (UniqueName: \"kubernetes.io/projected/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-kube-api-access-967xs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.727257 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.727227    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:31.790374 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.790342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.790507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.790395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.790557 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.790519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4vjj\" (UniqueName: \"kubernetes.io/projected/aae394bf-7dcf-4ecc-9013-2cb6130d6573-kube-api-access-s4vjj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.790798 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.790780    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.790848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.790835    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.798739 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.798713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4vjj\" (UniqueName: \"kubernetes.io/projected/aae394bf-7dcf-4ecc-9013-2cb6130d6573-kube-api-access-s4vjj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.825403 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.825374    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:31.902697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.902663    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"]
Sep 10 14:34:31.905896 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:34:31.905874    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec22522_2394_42ff_93fb_ac540aeed741.slice/crio-6578cc84b55e98f14cee073e6ce8b5edf094c62d1a7c1a54a1bb4788ce451e9d WatchSource:0}: Error finding container 6578cc84b55e98f14cee073e6ce8b5edf094c62d1a7c1a54a1bb4788ce451e9d: Status 404 returned error can't find the container with id 6578cc84b55e98f14cee073e6ce8b5edf094c62d1a7c1a54a1bb4788ce451e9d
Sep 10 14:34:31.933363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.933336    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:31.998429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:31.998397    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"]
Sep 10 14:34:32.000093 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:34:32.000069    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a1e10d_a33a_4700_a5c0_e9f63b9063c6.slice/crio-98d332d9216c52098686a4a934fdffab5239459c124a843d2637daf4602bd567 WatchSource:0}: Error finding container 98d332d9216c52098686a4a934fdffab5239459c124a843d2637daf4602bd567: Status 404 returned error can't find the container with id 98d332d9216c52098686a4a934fdffab5239459c124a843d2637daf4602bd567
Sep 10 14:34:32.097330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.096495    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"]
Sep 10 14:34:32.124036 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.123963    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr" event={"ID":"3e94bfa4-d891-4e7b-8541-ee5b8f02592d","Type":"ContainerStarted","Data":"b27d585c528c4c3644831ff7941dd189716c38dd13b35e6992903962eba63bde"}
Sep 10 14:34:32.125471 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.125444    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9" event={"ID":"60a1e10d-a33a-4700-a5c0-e9f63b9063c6","Type":"ContainerStarted","Data":"f56211114781fb2dcb5d3eb2f45dc02286525a9cf2b58d50e74c8ea5613ec9b3"}
Sep 10 14:34:32.125575 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.125493    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9" event={"ID":"60a1e10d-a33a-4700-a5c0-e9f63b9063c6","Type":"ContainerStarted","Data":"98d332d9216c52098686a4a934fdffab5239459c124a843d2637daf4602bd567"}
Sep 10 14:34:32.127004 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.126983    2560 generic.go:356] "Generic (PLEG): container finished" podID="8ec22522-2394-42ff-93fb-ac540aeed741" containerID="3d2f8bc4de1cf2ea83fa5bb0dffebc3dc7282d08ef3c8578be2fbd8a903d1ba7" exitCode=0
Sep 10 14:34:32.127090 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.127028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp" event={"ID":"8ec22522-2394-42ff-93fb-ac540aeed741","Type":"ContainerDied","Data":"3d2f8bc4de1cf2ea83fa5bb0dffebc3dc7282d08ef3c8578be2fbd8a903d1ba7"}
Sep 10 14:34:32.127090 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.127058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp" event={"ID":"8ec22522-2394-42ff-93fb-ac540aeed741","Type":"ContainerStarted","Data":"6578cc84b55e98f14cee073e6ce8b5edf094c62d1a7c1a54a1bb4788ce451e9d"}
Sep 10 14:34:32.204910 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:32.204877    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"]
Sep 10 14:34:32.206811 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:34:32.206787    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae394bf_7dcf_4ecc_9013_2cb6130d6573.slice/crio-f7af6a993b794d592a488b5ffba8b5031152e5ca59d4bfe533809ad337605db9 WatchSource:0}: Error finding container f7af6a993b794d592a488b5ffba8b5031152e5ca59d4bfe533809ad337605db9: Status 404 returned error can't find the container with id f7af6a993b794d592a488b5ffba8b5031152e5ca59d4bfe533809ad337605db9
Sep 10 14:34:33.132888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.132796    2560 generic.go:356] "Generic (PLEG): container finished" podID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerID="be077406131bee708e019c0d170dbf15f84d37d3b08040841d24e338c64097a1" exitCode=0
Sep 10 14:34:33.133259 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.132880    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr" event={"ID":"3e94bfa4-d891-4e7b-8541-ee5b8f02592d","Type":"ContainerDied","Data":"be077406131bee708e019c0d170dbf15f84d37d3b08040841d24e338c64097a1"}
Sep 10 14:34:33.134535 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.134509    2560 generic.go:356] "Generic (PLEG): container finished" podID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerID="f56211114781fb2dcb5d3eb2f45dc02286525a9cf2b58d50e74c8ea5613ec9b3" exitCode=0
Sep 10 14:34:33.134608 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.134581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9" event={"ID":"60a1e10d-a33a-4700-a5c0-e9f63b9063c6","Type":"ContainerDied","Data":"f56211114781fb2dcb5d3eb2f45dc02286525a9cf2b58d50e74c8ea5613ec9b3"}
Sep 10 14:34:33.136734 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.136708    2560 generic.go:356] "Generic (PLEG): container finished" podID="8ec22522-2394-42ff-93fb-ac540aeed741" containerID="549c9edc399b7904e2a1ae8c251a2ee9bb972bb914def332e00ccfe18f02ba01" exitCode=0
Sep 10 14:34:33.136824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.136777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp" event={"ID":"8ec22522-2394-42ff-93fb-ac540aeed741","Type":"ContainerDied","Data":"549c9edc399b7904e2a1ae8c251a2ee9bb972bb914def332e00ccfe18f02ba01"}
Sep 10 14:34:33.138456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.138433    2560 generic.go:356] "Generic (PLEG): container finished" podID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerID="e29fe6b18d1e2a66a6622fecca56a6ee87da4376bda6f0d917d46ad396390033" exitCode=0
Sep 10 14:34:33.138548 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.138462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj" event={"ID":"aae394bf-7dcf-4ecc-9013-2cb6130d6573","Type":"ContainerDied","Data":"e29fe6b18d1e2a66a6622fecca56a6ee87da4376bda6f0d917d46ad396390033"}
Sep 10 14:34:33.138548 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:33.138488    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj" event={"ID":"aae394bf-7dcf-4ecc-9013-2cb6130d6573","Type":"ContainerStarted","Data":"f7af6a993b794d592a488b5ffba8b5031152e5ca59d4bfe533809ad337605db9"}
Sep 10 14:34:34.145359 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:34.145324    2560 generic.go:356] "Generic (PLEG): container finished" podID="8ec22522-2394-42ff-93fb-ac540aeed741" containerID="cd87db0b3fcb9269c3343e0af8811f1540018a8dc2b77603b2157826972036d3" exitCode=0
Sep 10 14:34:34.145759 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:34.145423    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp" event={"ID":"8ec22522-2394-42ff-93fb-ac540aeed741","Type":"ContainerDied","Data":"cd87db0b3fcb9269c3343e0af8811f1540018a8dc2b77603b2157826972036d3"}
Sep 10 14:34:34.147261 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:34.147243    2560 generic.go:356] "Generic (PLEG): container finished" podID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerID="e7de4159c3c4511ce8726a077fc5e21dda11c14d09b6e85461c24aab88698238" exitCode=0
Sep 10 14:34:34.147375 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:34.147307    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr" event={"ID":"3e94bfa4-d891-4e7b-8541-ee5b8f02592d","Type":"ContainerDied","Data":"e7de4159c3c4511ce8726a077fc5e21dda11c14d09b6e85461c24aab88698238"}
Sep 10 14:34:35.154711 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.154679    2560 generic.go:356] "Generic (PLEG): container finished" podID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerID="968a4702c92a8409d17259c1b3ee902ba4339dea8d9a572dc7c0b00d7242e9df" exitCode=0
Sep 10 14:34:35.155118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.154765    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr" event={"ID":"3e94bfa4-d891-4e7b-8541-ee5b8f02592d","Type":"ContainerDied","Data":"968a4702c92a8409d17259c1b3ee902ba4339dea8d9a572dc7c0b00d7242e9df"}
Sep 10 14:34:35.161936 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.161911    2560 generic.go:356] "Generic (PLEG): container finished" podID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerID="a291c0cec239b45a1a6b4c1f8d1c7d521928884caa3171fc126dd76a3d1c4510" exitCode=0
Sep 10 14:34:35.162091 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.162014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9" event={"ID":"60a1e10d-a33a-4700-a5c0-e9f63b9063c6","Type":"ContainerDied","Data":"a291c0cec239b45a1a6b4c1f8d1c7d521928884caa3171fc126dd76a3d1c4510"}
Sep 10 14:34:35.164117 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.164097    2560 generic.go:356] "Generic (PLEG): container finished" podID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerID="f2b6853cb73938e82b37f50db9c03abf223222885d714854607496dcb11e3ed2" exitCode=0
Sep 10 14:34:35.164216 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.164152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj" event={"ID":"aae394bf-7dcf-4ecc-9013-2cb6130d6573","Type":"ContainerDied","Data":"f2b6853cb73938e82b37f50db9c03abf223222885d714854607496dcb11e3ed2"}
Sep 10 14:34:35.301266 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.301238    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:35.425468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.425424    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-bundle\") pod \"8ec22522-2394-42ff-93fb-ac540aeed741\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") "
Sep 10 14:34:35.425468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.425471    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-util\") pod \"8ec22522-2394-42ff-93fb-ac540aeed741\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") "
Sep 10 14:34:35.425702 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.425495    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hh7q\" (UniqueName: \"kubernetes.io/projected/8ec22522-2394-42ff-93fb-ac540aeed741-kube-api-access-6hh7q\") pod \"8ec22522-2394-42ff-93fb-ac540aeed741\" (UID: \"8ec22522-2394-42ff-93fb-ac540aeed741\") "
Sep 10 14:34:35.425972 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.425944    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-bundle" (OuterVolumeSpecName: "bundle") pod "8ec22522-2394-42ff-93fb-ac540aeed741" (UID: "8ec22522-2394-42ff-93fb-ac540aeed741"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:35.427702 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.427675    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec22522-2394-42ff-93fb-ac540aeed741-kube-api-access-6hh7q" (OuterVolumeSpecName: "kube-api-access-6hh7q") pod "8ec22522-2394-42ff-93fb-ac540aeed741" (UID: "8ec22522-2394-42ff-93fb-ac540aeed741"). InnerVolumeSpecName "kube-api-access-6hh7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:35.430691 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.430672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-util" (OuterVolumeSpecName: "util") pod "8ec22522-2394-42ff-93fb-ac540aeed741" (UID: "8ec22522-2394-42ff-93fb-ac540aeed741"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:35.526406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.526372    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:35.526406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.526401    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec22522-2394-42ff-93fb-ac540aeed741-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:35.526588 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:35.526413    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hh7q\" (UniqueName: \"kubernetes.io/projected/8ec22522-2394-42ff-93fb-ac540aeed741-kube-api-access-6hh7q\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:36.171685 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.171649    2560 generic.go:356] "Generic (PLEG): container finished" podID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerID="30652ac4edb942d4cf1c7a013a08a494be4d48d9a70053cb5fa6a7b6f00c4b46" exitCode=0
Sep 10 14:34:36.172114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.171733    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9" event={"ID":"60a1e10d-a33a-4700-a5c0-e9f63b9063c6","Type":"ContainerDied","Data":"30652ac4edb942d4cf1c7a013a08a494be4d48d9a70053cb5fa6a7b6f00c4b46"}
Sep 10 14:34:36.173613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.173594    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp"
Sep 10 14:34:36.173740 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.173610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p5kzp" event={"ID":"8ec22522-2394-42ff-93fb-ac540aeed741","Type":"ContainerDied","Data":"6578cc84b55e98f14cee073e6ce8b5edf094c62d1a7c1a54a1bb4788ce451e9d"}
Sep 10 14:34:36.173740 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.173644    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6578cc84b55e98f14cee073e6ce8b5edf094c62d1a7c1a54a1bb4788ce451e9d"
Sep 10 14:34:36.175996 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.175976    2560 generic.go:356] "Generic (PLEG): container finished" podID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerID="4c8c393f84fa15e67a6ca820885f8c1fad91c22d03d3e636fff9d5bdd21d2a87" exitCode=0
Sep 10 14:34:36.176081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.176053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj" event={"ID":"aae394bf-7dcf-4ecc-9013-2cb6130d6573","Type":"ContainerDied","Data":"4c8c393f84fa15e67a6ca820885f8c1fad91c22d03d3e636fff9d5bdd21d2a87"}
Sep 10 14:34:36.298673 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.298648    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:36.433506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.433400    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-bundle\") pod \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") "
Sep 10 14:34:36.433506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.433465    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-util\") pod \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") "
Sep 10 14:34:36.433506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.433511    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-967xs\" (UniqueName: \"kubernetes.io/projected/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-kube-api-access-967xs\") pod \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\" (UID: \"3e94bfa4-d891-4e7b-8541-ee5b8f02592d\") "
Sep 10 14:34:36.434006 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.433981    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-bundle" (OuterVolumeSpecName: "bundle") pod "3e94bfa4-d891-4e7b-8541-ee5b8f02592d" (UID: "3e94bfa4-d891-4e7b-8541-ee5b8f02592d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:36.435628 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.435605    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-kube-api-access-967xs" (OuterVolumeSpecName: "kube-api-access-967xs") pod "3e94bfa4-d891-4e7b-8541-ee5b8f02592d" (UID: "3e94bfa4-d891-4e7b-8541-ee5b8f02592d"). InnerVolumeSpecName "kube-api-access-967xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:36.438986 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.438958    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-util" (OuterVolumeSpecName: "util") pod "3e94bfa4-d891-4e7b-8541-ee5b8f02592d" (UID: "3e94bfa4-d891-4e7b-8541-ee5b8f02592d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:36.534572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.534537    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-967xs\" (UniqueName: \"kubernetes.io/projected/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-kube-api-access-967xs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:36.534572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.534567    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:36.534572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:36.534577    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e94bfa4-d891-4e7b-8541-ee5b8f02592d-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:37.183055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.183028    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr"
Sep 10 14:34:37.183470 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.183058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4zhgr" event={"ID":"3e94bfa4-d891-4e7b-8541-ee5b8f02592d","Type":"ContainerDied","Data":"b27d585c528c4c3644831ff7941dd189716c38dd13b35e6992903962eba63bde"}
Sep 10 14:34:37.183470 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.183104    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b27d585c528c4c3644831ff7941dd189716c38dd13b35e6992903962eba63bde"
Sep 10 14:34:37.309199 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.309172    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:37.344714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.344687    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:37.442086 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442058    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-bundle\") pod \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") "
Sep 10 14:34:37.442255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442107    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-bundle\") pod \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") "
Sep 10 14:34:37.442255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442139    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-util\") pod \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") "
Sep 10 14:34:37.442255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442169    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4vjj\" (UniqueName: \"kubernetes.io/projected/aae394bf-7dcf-4ecc-9013-2cb6130d6573-kube-api-access-s4vjj\") pod \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\" (UID: \"aae394bf-7dcf-4ecc-9013-2cb6130d6573\") "
Sep 10 14:34:37.442449 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442302    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbshl\" (UniqueName: \"kubernetes.io/projected/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-kube-api-access-nbshl\") pod \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") "
Sep 10 14:34:37.442449 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442346    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-util\") pod \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\" (UID: \"60a1e10d-a33a-4700-a5c0-e9f63b9063c6\") "
Sep 10 14:34:37.442821 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.442792    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-bundle" (OuterVolumeSpecName: "bundle") pod "60a1e10d-a33a-4700-a5c0-e9f63b9063c6" (UID: "60a1e10d-a33a-4700-a5c0-e9f63b9063c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:37.443034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.443012    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-bundle" (OuterVolumeSpecName: "bundle") pod "aae394bf-7dcf-4ecc-9013-2cb6130d6573" (UID: "aae394bf-7dcf-4ecc-9013-2cb6130d6573"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:37.444398 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.444363    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-kube-api-access-nbshl" (OuterVolumeSpecName: "kube-api-access-nbshl") pod "60a1e10d-a33a-4700-a5c0-e9f63b9063c6" (UID: "60a1e10d-a33a-4700-a5c0-e9f63b9063c6"). InnerVolumeSpecName "kube-api-access-nbshl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:37.444512 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.444398    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae394bf-7dcf-4ecc-9013-2cb6130d6573-kube-api-access-s4vjj" (OuterVolumeSpecName: "kube-api-access-s4vjj") pod "aae394bf-7dcf-4ecc-9013-2cb6130d6573" (UID: "aae394bf-7dcf-4ecc-9013-2cb6130d6573"). InnerVolumeSpecName "kube-api-access-s4vjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:37.448012 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.447977    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-util" (OuterVolumeSpecName: "util") pod "aae394bf-7dcf-4ecc-9013-2cb6130d6573" (UID: "aae394bf-7dcf-4ecc-9013-2cb6130d6573"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:37.448240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.448219    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-util" (OuterVolumeSpecName: "util") pod "60a1e10d-a33a-4700-a5c0-e9f63b9063c6" (UID: "60a1e10d-a33a-4700-a5c0-e9f63b9063c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:37.543614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.543575    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbshl\" (UniqueName: \"kubernetes.io/projected/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-kube-api-access-nbshl\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:37.543614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.543605    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:37.543614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.543614    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60a1e10d-a33a-4700-a5c0-e9f63b9063c6-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:37.543614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.543623    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-bundle\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:37.543875 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.543631    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae394bf-7dcf-4ecc-9013-2cb6130d6573-util\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:37.543875 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:37.543641    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4vjj\" (UniqueName: \"kubernetes.io/projected/aae394bf-7dcf-4ecc-9013-2cb6130d6573-kube-api-access-s4vjj\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:34:38.195222 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:38.195189    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9"
Sep 10 14:34:38.195222 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:38.195212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s4rq9" event={"ID":"60a1e10d-a33a-4700-a5c0-e9f63b9063c6","Type":"ContainerDied","Data":"98d332d9216c52098686a4a934fdffab5239459c124a843d2637daf4602bd567"}
Sep 10 14:34:38.195715 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:38.195304    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98d332d9216c52098686a4a934fdffab5239459c124a843d2637daf4602bd567"
Sep 10 14:34:38.197321 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:38.197295    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj" event={"ID":"aae394bf-7dcf-4ecc-9013-2cb6130d6573","Type":"ContainerDied","Data":"f7af6a993b794d592a488b5ffba8b5031152e5ca59d4bfe533809ad337605db9"}
Sep 10 14:34:38.197436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:38.197334    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7af6a993b794d592a488b5ffba8b5031152e5ca59d4bfe533809ad337605db9"
Sep 10 14:34:38.197436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:38.197356    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7trxj"
Sep 10 14:34:42.208446 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:42.208417    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:44.261265 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261227    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"]
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261530    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="pull"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261542    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="pull"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261551    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="pull"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261557    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="pull"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261562    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261568    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261578    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261583    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261589    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261593    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261600    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261605    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261612    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261617    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261624    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261628    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261634    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="pull"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261639    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="pull"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261650    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261655    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="util"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261662    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="extract"
Sep 10 14:34:44.261659 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261667    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="extract"
Sep 10 14:34:44.262380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261674    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="pull"
Sep 10 14:34:44.262380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261679    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="pull"
Sep 10 14:34:44.262380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261727    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e94bfa4-d891-4e7b-8541-ee5b8f02592d" containerName="extract"
Sep 10 14:34:44.262380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261736    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec22522-2394-42ff-93fb-ac540aeed741" containerName="extract"
Sep 10 14:34:44.262380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261741    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="aae394bf-7dcf-4ecc-9013-2cb6130d6573" containerName="extract"
Sep 10 14:34:44.262380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.261747    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a1e10d-a33a-4700-a5c0-e9f63b9063c6" containerName="extract"
Sep 10 14:34:44.264653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.264636    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:34:44.267348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.267325    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 14:34:44.267477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.267326    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-z42tz\""
Sep 10 14:34:44.267866 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.267853    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 14:34:44.280675 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.280646    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"]
Sep 10 14:34:44.399592 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.399556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mwfs\" (UniqueName: \"kubernetes.io/projected/32ca470e-5b0f-4887-8d29-bf30e2270de7-kube-api-access-6mwfs\") pod \"limitador-operator-controller-manager-74495c69f8-2wtl8\" (UID: \"32ca470e-5b0f-4887-8d29-bf30e2270de7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:34:44.501273 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.501235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mwfs\" (UniqueName: \"kubernetes.io/projected/32ca470e-5b0f-4887-8d29-bf30e2270de7-kube-api-access-6mwfs\") pod \"limitador-operator-controller-manager-74495c69f8-2wtl8\" (UID: \"32ca470e-5b0f-4887-8d29-bf30e2270de7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:34:44.511694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.511573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mwfs\" (UniqueName: \"kubernetes.io/projected/32ca470e-5b0f-4887-8d29-bf30e2270de7-kube-api-access-6mwfs\") pod \"limitador-operator-controller-manager-74495c69f8-2wtl8\" (UID: \"32ca470e-5b0f-4887-8d29-bf30e2270de7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:34:44.575406 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.575376    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:34:44.615335 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.615253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:44.860130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:44.860103    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"]
Sep 10 14:34:44.862386 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:34:44.862358    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ca470e_5b0f_4887_8d29_bf30e2270de7.slice/crio-4d2e4f53a0e585d095dc4c2f3e6ae3f7cfcca79d90a7ee6fa6e966d80baa90fd WatchSource:0}: Error finding container 4d2e4f53a0e585d095dc4c2f3e6ae3f7cfcca79d90a7ee6fa6e966d80baa90fd: Status 404 returned error can't find the container with id 4d2e4f53a0e585d095dc4c2f3e6ae3f7cfcca79d90a7ee6fa6e966d80baa90fd
Sep 10 14:34:45.044655 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:45.044611    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:45.236358 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:45.236322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8" event={"ID":"32ca470e-5b0f-4887-8d29-bf30e2270de7","Type":"ContainerStarted","Data":"4d2e4f53a0e585d095dc4c2f3e6ae3f7cfcca79d90a7ee6fa6e966d80baa90fd"}
Sep 10 14:34:46.816177 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:46.816144    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-c5gtp"]
Sep 10 14:34:46.826430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:46.826398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:34:46.829689 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:46.829453    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9tbnv\""
Sep 10 14:34:46.833676 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:46.833137    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-c5gtp"]
Sep 10 14:34:46.922368 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:46.922323    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz8xz\" (UniqueName: \"kubernetes.io/projected/72bb8b69-9b7c-4f24-89f7-00124fe570ab-kube-api-access-rz8xz\") pod \"authorino-operator-76b478bd78-c5gtp\" (UID: \"72bb8b69-9b7c-4f24-89f7-00124fe570ab\") " pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:34:47.023714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:47.023681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz8xz\" (UniqueName: \"kubernetes.io/projected/72bb8b69-9b7c-4f24-89f7-00124fe570ab-kube-api-access-rz8xz\") pod \"authorino-operator-76b478bd78-c5gtp\" (UID: \"72bb8b69-9b7c-4f24-89f7-00124fe570ab\") " pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:34:47.036680 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:47.036654    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz8xz\" (UniqueName: \"kubernetes.io/projected/72bb8b69-9b7c-4f24-89f7-00124fe570ab-kube-api-access-rz8xz\") pod \"authorino-operator-76b478bd78-c5gtp\" (UID: \"72bb8b69-9b7c-4f24-89f7-00124fe570ab\") " pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:34:47.140518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:47.140441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:34:47.263028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:47.262998    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:47.520369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:47.520335    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-c5gtp"]
Sep 10 14:34:47.522564 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:34:47.522531    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bb8b69_9b7c_4f24_89f7_00124fe570ab.slice/crio-f3055d738ef3d274aac45b809603f552a8b842a10e06e31bcc6a51f2c71e16d6 WatchSource:0}: Error finding container f3055d738ef3d274aac45b809603f552a8b842a10e06e31bcc6a51f2c71e16d6: Status 404 returned error can't find the container with id f3055d738ef3d274aac45b809603f552a8b842a10e06e31bcc6a51f2c71e16d6
Sep 10 14:34:47.867616 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:47.867585    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:48.255102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:48.255062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp" event={"ID":"72bb8b69-9b7c-4f24-89f7-00124fe570ab","Type":"ContainerStarted","Data":"f3055d738ef3d274aac45b809603f552a8b842a10e06e31bcc6a51f2c71e16d6"}
Sep 10 14:34:48.257088 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:48.257052    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8" event={"ID":"32ca470e-5b0f-4887-8d29-bf30e2270de7","Type":"ContainerStarted","Data":"547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed"}
Sep 10 14:34:48.257235 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:48.257167    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:34:48.277369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:48.277317    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8" podStartSLOduration=1.822444411 podStartE2EDuration="4.277301194s" totalImagesPullingTime="2.454856783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:48.275184329 +0000 UTC m=+417.630997251" watchObservedRunningTime="2026-09-10 14:34:48.277301194 +0000 UTC m=+417.633114092"
Sep 10 14:34:48.630698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:48.630613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:49.013183 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.013148    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"]
Sep 10 14:34:49.016474 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.016458    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:34:49.019069 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.019046    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 14:34:49.019182 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.019046    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hqzcf\""
Sep 10 14:34:49.038734 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.038709    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"]
Sep 10 14:34:49.142060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.142025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9pzj\" (UniqueName: \"kubernetes.io/projected/5f485167-8a1a-494a-bebb-5a8ec3c8fff2-kube-api-access-l9pzj\") pod \"dns-operator-controller-manager-7b84cd7c9c-2jjlc\" (UID: \"5f485167-8a1a-494a-bebb-5a8ec3c8fff2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:34:49.243365 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.243331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9pzj\" (UniqueName: \"kubernetes.io/projected/5f485167-8a1a-494a-bebb-5a8ec3c8fff2-kube-api-access-l9pzj\") pod \"dns-operator-controller-manager-7b84cd7c9c-2jjlc\" (UID: \"5f485167-8a1a-494a-bebb-5a8ec3c8fff2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:34:49.255653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.255620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9pzj\" (UniqueName: \"kubernetes.io/projected/5f485167-8a1a-494a-bebb-5a8ec3c8fff2-kube-api-access-l9pzj\") pod \"dns-operator-controller-manager-7b84cd7c9c-2jjlc\" (UID: \"5f485167-8a1a-494a-bebb-5a8ec3c8fff2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:34:49.264086 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.264027    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp" event={"ID":"72bb8b69-9b7c-4f24-89f7-00124fe570ab","Type":"ContainerStarted","Data":"e344e74b49d749b1311ced44f2fdbdcdd187d898f996b1cd1116dfb5c6b28cf6"}
Sep 10 14:34:49.264290 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.264259    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:34:49.286693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.286642    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp" podStartSLOduration=2.11083811 podStartE2EDuration="3.286625576s" totalImagesPullingTime="1.175787466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:49.285713023 +0000 UTC m=+418.641525953" watchObservedRunningTime="2026-09-10 14:34:49.286625576 +0000 UTC m=+418.642438486"
Sep 10 14:34:49.325747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.325721    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:34:49.599783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.599752    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"]
Sep 10 14:34:49.601479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.601435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:49.601609 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:34:49.601585    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f485167_8a1a_494a_bebb_5a8ec3c8fff2.slice/crio-fd8bbfd850db06aa5704a5bb1d6be1b6ed4d6cd4207a2752a775869c244100cb WatchSource:0}: Error finding container fd8bbfd850db06aa5704a5bb1d6be1b6ed4d6cd4207a2752a775869c244100cb: Status 404 returned error can't find the container with id fd8bbfd850db06aa5704a5bb1d6be1b6ed4d6cd4207a2752a775869c244100cb
Sep 10 14:34:49.981409 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:49.981375    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:50.218127 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:50.218092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:50.270363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:50.270273    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc" event={"ID":"5f485167-8a1a-494a-bebb-5a8ec3c8fff2","Type":"ContainerStarted","Data":"fd8bbfd850db06aa5704a5bb1d6be1b6ed4d6cd4207a2752a775869c244100cb"}
Sep 10 14:34:51.089677 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:51.089642    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:51.268960 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:51.268935    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:51.309153 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:51.309131    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 14:34:52.284479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:52.284440    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc" event={"ID":"5f485167-8a1a-494a-bebb-5a8ec3c8fff2","Type":"ContainerStarted","Data":"af645c585f952926fc87f73730f2e6bcd0e89f684318d77c6ade2952895a38e8"}
Sep 10 14:34:52.284924 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:52.284588    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:34:52.305163 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:52.305096    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc" podStartSLOduration=2.602754087 podStartE2EDuration="4.305077417s" totalImagesPullingTime="1.70232333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:52.301615978 +0000 UTC m=+421.657428905" watchObservedRunningTime="2026-09-10 14:34:52.305077417 +0000 UTC m=+421.660890324"
Sep 10 14:34:57.236014 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:57.235980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:57.341327 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:57.341275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:59.267012 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:34:59.266982    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:00.272869 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:00.272836    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-c5gtp"
Sep 10 14:35:01.889339 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.889269    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:01.966558 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.966525    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:01.968962 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.968938    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"]
Sep 10 14:35:01.972720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.972696    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:01.975090 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.975065    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nfwh2\""
Sep 10 14:35:01.975210 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.975113    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 14:35:01.975427 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.975401    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 14:35:01.980986 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:01.980963    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"]
Sep 10 14:35:02.149870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.149783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cdeac31-1b97-4dad-91ae-4e9d95015421-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.149870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.149839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgncf\" (UniqueName: \"kubernetes.io/projected/8cdeac31-1b97-4dad-91ae-4e9d95015421-kube-api-access-jgncf\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.150054 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.149949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdeac31-1b97-4dad-91ae-4e9d95015421-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.220858 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.220821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:02.251038 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.251000    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdeac31-1b97-4dad-91ae-4e9d95015421-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.251223 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.251064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cdeac31-1b97-4dad-91ae-4e9d95015421-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.251223 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.251113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgncf\" (UniqueName: \"kubernetes.io/projected/8cdeac31-1b97-4dad-91ae-4e9d95015421-kube-api-access-jgncf\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.251223 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:02.251152    2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 10 14:35:02.251428 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:02.251248    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cdeac31-1b97-4dad-91ae-4e9d95015421-plugin-serving-cert podName:8cdeac31-1b97-4dad-91ae-4e9d95015421 nodeName:}" failed. No retries permitted until 2026-09-10 14:35:02.751225489 +0000 UTC m=+432.107038387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/8cdeac31-1b97-4dad-91ae-4e9d95015421-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-w5lpc" (UID: "8cdeac31-1b97-4dad-91ae-4e9d95015421") : secret "plugin-serving-cert" not found
Sep 10 14:35:02.251774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.251754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cdeac31-1b97-4dad-91ae-4e9d95015421-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.259744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.259719    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgncf\" (UniqueName: \"kubernetes.io/projected/8cdeac31-1b97-4dad-91ae-4e9d95015421-kube-api-access-jgncf\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.384488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.384442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:02.484846 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.484813    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:02.755067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.754971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdeac31-1b97-4dad-91ae-4e9d95015421-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.757483 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.757458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdeac31-1b97-4dad-91ae-4e9d95015421-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-w5lpc\" (UID: \"8cdeac31-1b97-4dad-91ae-4e9d95015421\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:02.882458 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:02.882419    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"
Sep 10 14:35:03.155258 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:03.155227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc"]
Sep 10 14:35:03.157107 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:03.157078    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cdeac31_1b97_4dad_91ae_4e9d95015421.slice/crio-1740f98b5a1de28f25959e8b911b54afb4a1e9a0f4983c0aaba7edd483f35b04 WatchSource:0}: Error finding container 1740f98b5a1de28f25959e8b911b54afb4a1e9a0f4983c0aaba7edd483f35b04: Status 404 returned error can't find the container with id 1740f98b5a1de28f25959e8b911b54afb4a1e9a0f4983c0aaba7edd483f35b04
Sep 10 14:35:03.292560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:03.292526    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2jjlc"
Sep 10 14:35:03.341001 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:03.340960    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc" event={"ID":"8cdeac31-1b97-4dad-91ae-4e9d95015421","Type":"ContainerStarted","Data":"1740f98b5a1de28f25959e8b911b54afb4a1e9a0f4983c0aaba7edd483f35b04"}
Sep 10 14:35:06.206518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:06.206473    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:06.271985 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:06.271943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:08.370796 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:08.370761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc" event={"ID":"8cdeac31-1b97-4dad-91ae-4e9d95015421","Type":"ContainerStarted","Data":"36ee648984a4cc8efe7d1ffadc2faee45c4ed1da8daf567dac857fd813186982"}
Sep 10 14:35:08.387884 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:08.387823    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-w5lpc" podStartSLOduration=3.059653781 podStartE2EDuration="7.387805715s" totalImagesPullingTime="4.328151934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:08.386326804 +0000 UTC m=+437.742139731" watchObservedRunningTime="2026-09-10 14:35:08.387805715 +0000 UTC m=+437.743618622"
Sep 10 14:35:11.866625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:11.866585    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:12.058171 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.058125    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:12.260844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.260812    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"]
Sep 10 14:35:12.265744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.265723    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.268052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.268032    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kkwjv\""
Sep 10 14:35:12.275328 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.275307    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"]
Sep 10 14:35:12.339595 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.339551    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.339766 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.339658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmnpv\" (UniqueName: \"kubernetes.io/projected/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-kube-api-access-gmnpv\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.424495 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.422627    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:12.440942 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.440915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmnpv\" (UniqueName: \"kubernetes.io/projected/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-kube-api-access-gmnpv\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.441048 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.440977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.441413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.441393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.452800 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.452771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmnpv\" (UniqueName: \"kubernetes.io/projected/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-kube-api-access-gmnpv\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.526940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.526858    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:12.576084 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.576056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:12.851959 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:12.851935    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"]
Sep 10 14:35:12.854033 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:12.854004    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4dc4bd2_9ee0_4656_a353_84f0a94e6797.slice/crio-f9885efd70dea0be739b264e4dabe42834ceea024f4a8f9ea13542ba4f698f24 WatchSource:0}: Error finding container f9885efd70dea0be739b264e4dabe42834ceea024f4a8f9ea13542ba4f698f24: Status 404 returned error can't find the container with id f9885efd70dea0be739b264e4dabe42834ceea024f4a8f9ea13542ba4f698f24
Sep 10 14:35:13.018465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.018435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:13.120837 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.120752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:13.377102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.376993    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"]
Sep 10 14:35:13.382617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.382591    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"]
Sep 10 14:35:13.406411 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.406104    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"]
Sep 10 14:35:13.411374 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.411320    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.417650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.417622    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"]
Sep 10 14:35:13.417840 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.417807    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" containerName="manager" containerID="cri-o://547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed" gracePeriod=2
Sep 10 14:35:13.422925 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.422900    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"]
Sep 10 14:35:13.437869 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.437837    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"]
Sep 10 14:35:13.438883 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:13.438741    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2wtl8\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:13.449565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.449529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffb38220-c83f-430a-b719-bfaae4663d03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-dmktg\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.449705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.449682    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnntb\" (UniqueName: \"kubernetes.io/projected/ffb38220-c83f-430a-b719-bfaae4663d03-kube-api-access-bnntb\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-dmktg\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.450521 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.450499    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"]
Sep 10 14:35:13.450995 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.450977    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" containerName="manager"
Sep 10 14:35:13.451038 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.451000    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" containerName="manager"
Sep 10 14:35:13.451120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.451108    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" containerName="manager"
Sep 10 14:35:13.454444 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.454427    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:13.467205 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.467168    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"]
Sep 10 14:35:13.485457 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:13.485417    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2wtl8\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:13.550411 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.550374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnntb\" (UniqueName: \"kubernetes.io/projected/ffb38220-c83f-430a-b719-bfaae4663d03-kube-api-access-bnntb\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-dmktg\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.550587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.550432    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffb38220-c83f-430a-b719-bfaae4663d03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-dmktg\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.550587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.550465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmrkn\" (UniqueName: \"kubernetes.io/projected/4e0fa36a-408a-4c5b-92ca-1d01f0273eed-kube-api-access-xmrkn\") pod \"limitador-operator-controller-manager-74495c69f8-7qwrx\" (UID: \"4e0fa36a-408a-4c5b-92ca-1d01f0273eed\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:13.550884 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.550855    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffb38220-c83f-430a-b719-bfaae4663d03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-dmktg\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.569562 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.569525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnntb\" (UniqueName: \"kubernetes.io/projected/ffb38220-c83f-430a-b719-bfaae4663d03-kube-api-access-bnntb\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-dmktg\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.651738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.651708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmrkn\" (UniqueName: \"kubernetes.io/projected/4e0fa36a-408a-4c5b-92ca-1d01f0273eed-kube-api-access-xmrkn\") pod \"limitador-operator-controller-manager-74495c69f8-7qwrx\" (UID: \"4e0fa36a-408a-4c5b-92ca-1d01f0273eed\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:13.661201 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.661174    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:13.662654 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.662631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmrkn\" (UniqueName: \"kubernetes.io/projected/4e0fa36a-408a-4c5b-92ca-1d01f0273eed-kube-api-access-xmrkn\") pod \"limitador-operator-controller-manager-74495c69f8-7qwrx\" (UID: \"4e0fa36a-408a-4c5b-92ca-1d01f0273eed\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:13.663300 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:13.663257    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2wtl8\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:13.723887 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.723852    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:13.752094 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.752064    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mwfs\" (UniqueName: \"kubernetes.io/projected/32ca470e-5b0f-4887-8d29-bf30e2270de7-kube-api-access-6mwfs\") pod \"32ca470e-5b0f-4887-8d29-bf30e2270de7\" (UID: \"32ca470e-5b0f-4887-8d29-bf30e2270de7\") "
Sep 10 14:35:13.754907 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.754867    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32ca470e-5b0f-4887-8d29-bf30e2270de7-kube-api-access-6mwfs" (OuterVolumeSpecName: "kube-api-access-6mwfs") pod "32ca470e-5b0f-4887-8d29-bf30e2270de7" (UID: "32ca470e-5b0f-4887-8d29-bf30e2270de7"). InnerVolumeSpecName "kube-api-access-6mwfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:13.797801 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.797734    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:13.854389 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:13.854360    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mwfs\" (UniqueName: \"kubernetes.io/projected/32ca470e-5b0f-4887-8d29-bf30e2270de7-kube-api-access-6mwfs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:35:14.011111 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.011084    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"]
Sep 10 14:35:14.013213 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:14.013187    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb38220_c83f_430a_b719_bfaae4663d03.slice/crio-6c2dc2deaeffb384ffa31244c0c5a0b80951e8155b78e66aa12bf11790cdc36e WatchSource:0}: Error finding container 6c2dc2deaeffb384ffa31244c0c5a0b80951e8155b78e66aa12bf11790cdc36e: Status 404 returned error can't find the container with id 6c2dc2deaeffb384ffa31244c0c5a0b80951e8155b78e66aa12bf11790cdc36e
Sep 10 14:35:14.089520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.089490    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"]
Sep 10 14:35:14.089948 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:14.089910    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0fa36a_408a_4c5b_92ca_1d01f0273eed.slice/crio-40446dc33e0d20931ab80106e59180a8f7765e31052260d451b2128397b341e3 WatchSource:0}: Error finding container 40446dc33e0d20931ab80106e59180a8f7765e31052260d451b2128397b341e3: Status 404 returned error can't find the container with id 40446dc33e0d20931ab80106e59180a8f7765e31052260d451b2128397b341e3
Sep 10 14:35:14.408172 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.408078    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx" event={"ID":"4e0fa36a-408a-4c5b-92ca-1d01f0273eed","Type":"ContainerStarted","Data":"5ef4c95b111c38d6fff2251880f499b2c6e6162af7689a8fe8c77661d12f1a48"}
Sep 10 14:35:14.408172 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.408141    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx" event={"ID":"4e0fa36a-408a-4c5b-92ca-1d01f0273eed","Type":"ContainerStarted","Data":"40446dc33e0d20931ab80106e59180a8f7765e31052260d451b2128397b341e3"}
Sep 10 14:35:14.408436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.408205    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:14.409779 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.409749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg" event={"ID":"ffb38220-c83f-430a-b719-bfaae4663d03","Type":"ContainerStarted","Data":"6c2dc2deaeffb384ffa31244c0c5a0b80951e8155b78e66aa12bf11790cdc36e"}
Sep 10 14:35:14.411871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.411841    2560 generic.go:356] "Generic (PLEG): container finished" podID="32ca470e-5b0f-4887-8d29-bf30e2270de7" containerID="547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed" exitCode=0
Sep 10 14:35:14.411968 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.411897    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:14.411968 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.411921    2560 scope.go:117] "RemoveContainer" containerID="547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed"
Sep 10 14:35:14.423003 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.422981    2560 scope.go:117] "RemoveContainer" containerID="547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed"
Sep 10 14:35:14.423471 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:14.423439    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed\": container with ID starting with 547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed not found: ID does not exist" containerID="547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed"
Sep 10 14:35:14.423580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.423482    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed"} err="failed to get container status \"547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed\": rpc error: code = NotFound desc = could not find container \"547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed\": container with ID starting with 547fed3920d827458a20a1009c059e242caa462701a37532573907f27d40a4ed not found: ID does not exist"
Sep 10 14:35:14.430683 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:14.430630    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx" podStartSLOduration=1.430611767 podStartE2EDuration="1.430611767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:14.428564039 +0000 UTC m=+443.784376970" watchObservedRunningTime="2026-09-10 14:35:14.430611767 +0000 UTC m=+443.786424674"
Sep 10 14:35:14.430807 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:14.430781    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2wtl8\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:14.432502 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:14.432470    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2wtl8\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2wtl8"
Sep 10 14:35:15.320627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:15.320590    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32ca470e-5b0f-4887-8d29-bf30e2270de7" path="/var/lib/kubelet/pods/32ca470e-5b0f-4887-8d29-bf30e2270de7/volumes"
Sep 10 14:35:17.434689 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.434597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg" event={"ID":"ffb38220-c83f-430a-b719-bfaae4663d03","Type":"ContainerStarted","Data":"c9dc0002602abaf6911798bd0c538086fe080ac3fdea193e3fdfcf9d140e5cbb"}
Sep 10 14:35:17.435066 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.434693    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:17.436445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.436420    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" containerName="manager" containerID="cri-o://06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795" gracePeriod=2
Sep 10 14:35:17.485720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.485660    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg" podStartSLOduration=1.475436893 podStartE2EDuration="4.485637615s" totalImagesPullingTime="3.010200722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:17.485038497 +0000 UTC m=+446.840851423" watchObservedRunningTime="2026-09-10 14:35:17.485637615 +0000 UTC m=+446.841450522"
Sep 10 14:35:17.673985 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.673958    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:17.676516 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:17.676490    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:17.683725 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.683702    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmnpv\" (UniqueName: \"kubernetes.io/projected/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-kube-api-access-gmnpv\") pod \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") "
Sep 10 14:35:17.683843 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.683735    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-extensions-socket-volume\") pod \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\" (UID: \"e4dc4bd2-9ee0-4656-a353-84f0a94e6797\") "
Sep 10 14:35:17.684045 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.684017    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e4dc4bd2-9ee0-4656-a353-84f0a94e6797" (UID: "e4dc4bd2-9ee0-4656-a353-84f0a94e6797"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:17.685746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.685700    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-kube-api-access-gmnpv" (OuterVolumeSpecName: "kube-api-access-gmnpv") pod "e4dc4bd2-9ee0-4656-a353-84f0a94e6797" (UID: "e4dc4bd2-9ee0-4656-a353-84f0a94e6797"). InnerVolumeSpecName "kube-api-access-gmnpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:17.784768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.784731    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmnpv\" (UniqueName: \"kubernetes.io/projected/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-kube-api-access-gmnpv\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:35:17.784768 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:17.784762    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4dc4bd2-9ee0-4656-a353-84f0a94e6797-extensions-socket-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:35:18.449687 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:18.449652    2560 generic.go:356] "Generic (PLEG): container finished" podID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" containerID="06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795" exitCode=2
Sep 10 14:35:18.450190 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:18.449702    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:18.450190 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:18.449765    2560 scope.go:117] "RemoveContainer" containerID="06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795"
Sep 10 14:35:18.452185 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:18.452157    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:18.459610 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:18.459594    2560 scope.go:117] "RemoveContainer" containerID="06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795"
Sep 10 14:35:18.459887 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:18.459863    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795\": container with ID starting with 06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795 not found: ID does not exist" containerID="06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795"
Sep 10 14:35:18.459939 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:18.459890    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795"} err="failed to get container status \"06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795\": rpc error: code = NotFound desc = could not find container \"06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795\": container with ID starting with 06884ac453a786e99bc2852bcff2530072a7e9dd555e9ff6aff4185c88eac795 not found: ID does not exist"
Sep 10 14:35:18.460113 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:35:18.460094    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-9dbx7\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-9dbx7"
Sep 10 14:35:19.317739 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:19.317704    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" path="/var/lib/kubelet/pods/e4dc4bd2-9ee0-4656-a353-84f0a94e6797/volumes"
Sep 10 14:35:21.804568 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:21.804532    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:21.934345 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:21.934311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:25.424420 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:25.424385    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7qwrx"
Sep 10 14:35:26.613824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:26.613790    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:26.714250 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:26.714213    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:27.015206 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:27.015171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:27.115524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:27.115488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:28.452335 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:28.452304    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:30.625757 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:30.625724    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:30.734069 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:30.734036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:35.362262 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:35.362229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:35.460245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:35.460214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:35.560596 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:35.560562    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:41.352248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:41.352213    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"]
Sep 10 14:35:41.352754 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:41.352418    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg" podUID="ffb38220-c83f-430a-b719-bfaae4663d03" containerName="manager" containerID="cri-o://c9dc0002602abaf6911798bd0c538086fe080ac3fdea193e3fdfcf9d140e5cbb" gracePeriod=10
Sep 10 14:35:41.584479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:41.584446    2560 generic.go:356] "Generic (PLEG): container finished" podID="ffb38220-c83f-430a-b719-bfaae4663d03" containerID="c9dc0002602abaf6911798bd0c538086fe080ac3fdea193e3fdfcf9d140e5cbb" exitCode=0
Sep 10 14:35:41.584626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:41.584518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg" event={"ID":"ffb38220-c83f-430a-b719-bfaae4663d03","Type":"ContainerDied","Data":"c9dc0002602abaf6911798bd0c538086fe080ac3fdea193e3fdfcf9d140e5cbb"}
Sep 10 14:35:42.098377 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.098354    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:42.158205 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.158175    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffb38220-c83f-430a-b719-bfaae4663d03-extensions-socket-volume\") pod \"ffb38220-c83f-430a-b719-bfaae4663d03\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") "
Sep 10 14:35:42.158403 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.158242    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnntb\" (UniqueName: \"kubernetes.io/projected/ffb38220-c83f-430a-b719-bfaae4663d03-kube-api-access-bnntb\") pod \"ffb38220-c83f-430a-b719-bfaae4663d03\" (UID: \"ffb38220-c83f-430a-b719-bfaae4663d03\") "
Sep 10 14:35:42.158682 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.158657    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb38220-c83f-430a-b719-bfaae4663d03-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ffb38220-c83f-430a-b719-bfaae4663d03" (UID: "ffb38220-c83f-430a-b719-bfaae4663d03"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:42.160290 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.160253    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb38220-c83f-430a-b719-bfaae4663d03-kube-api-access-bnntb" (OuterVolumeSpecName: "kube-api-access-bnntb") pod "ffb38220-c83f-430a-b719-bfaae4663d03" (UID: "ffb38220-c83f-430a-b719-bfaae4663d03"). InnerVolumeSpecName "kube-api-access-bnntb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:42.259774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.259682    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffb38220-c83f-430a-b719-bfaae4663d03-extensions-socket-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:35:42.259774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.259717    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnntb\" (UniqueName: \"kubernetes.io/projected/ffb38220-c83f-430a-b719-bfaae4663d03-kube-api-access-bnntb\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:35:42.590948 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.590867    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"
Sep 10 14:35:42.590948 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.590910    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg" event={"ID":"ffb38220-c83f-430a-b719-bfaae4663d03","Type":"ContainerDied","Data":"6c2dc2deaeffb384ffa31244c0c5a0b80951e8155b78e66aa12bf11790cdc36e"}
Sep 10 14:35:42.591398 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.590973    2560 scope.go:117] "RemoveContainer" containerID="c9dc0002602abaf6911798bd0c538086fe080ac3fdea193e3fdfcf9d140e5cbb"
Sep 10 14:35:42.614093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.614062    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"]
Sep 10 14:35:42.617842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:42.617818    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-dmktg"]
Sep 10 14:35:43.318166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:43.318133    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb38220-c83f-430a-b719-bfaae4663d03" path="/var/lib/kubelet/pods/ffb38220-c83f-430a-b719-bfaae4663d03/volumes"
Sep 10 14:35:45.015319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:45.015262    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:45.392601 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:45.392524    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:49.259077 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:49.259044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:49.359641 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:49.359602    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:53.969887 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:53.969848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:54.020940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:54.020904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:58.068951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.068914    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:58.070013 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.069988    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:35:58.070428 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.070405    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" containerName="manager"
Sep 10 14:35:58.070554 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.070517    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" containerName="manager"
Sep 10 14:35:58.070554 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.070546    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb38220-c83f-430a-b719-bfaae4663d03" containerName="manager"
Sep 10 14:35:58.070663 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.070553    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb38220-c83f-430a-b719-bfaae4663d03" containerName="manager"
Sep 10 14:35:58.070717 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.070677    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4dc4bd2-9ee0-4656-a353-84f0a94e6797" containerName="manager"
Sep 10 14:35:58.070717 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.070693    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb38220-c83f-430a-b719-bfaae4663d03" containerName="manager"
Sep 10 14:35:58.074872 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.074852    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.077789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.077768    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 14:35:58.082013 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.081723    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:35:58.179881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.179843    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:35:58.185125 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.185097    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47224bbe-ff14-4459-8616-4f0390fe10e2-config-file\") pod \"limitador-limitador-8588749977-cqz2v\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") " pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.185267 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.185229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmfng\" (UniqueName: \"kubernetes.io/projected/47224bbe-ff14-4459-8616-4f0390fe10e2-kube-api-access-nmfng\") pod \"limitador-limitador-8588749977-cqz2v\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") " pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.243373 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.243340    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:58.286369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.286337    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47224bbe-ff14-4459-8616-4f0390fe10e2-config-file\") pod \"limitador-limitador-8588749977-cqz2v\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") " pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.286528 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.286398    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmfng\" (UniqueName: \"kubernetes.io/projected/47224bbe-ff14-4459-8616-4f0390fe10e2-kube-api-access-nmfng\") pod \"limitador-limitador-8588749977-cqz2v\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") " pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.287023 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.286990    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47224bbe-ff14-4459-8616-4f0390fe10e2-config-file\") pod \"limitador-limitador-8588749977-cqz2v\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") " pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.300021 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.299992    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmfng\" (UniqueName: \"kubernetes.io/projected/47224bbe-ff14-4459-8616-4f0390fe10e2-kube-api-access-nmfng\") pod \"limitador-limitador-8588749977-cqz2v\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") " pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.333089 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.333021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:58.385595 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.385559    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:58.441034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.440747    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:35:58.445498 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.445415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.454072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.454044    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:35:58.487397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.487363    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:35:58.487680 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.487658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68ca634a-fcb0-4dda-8e1b-12829c81221a-config-file\") pod \"limitador-limitador-84994f7ffc-v24tf\" (UID: \"68ca634a-fcb0-4dda-8e1b-12829c81221a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.487729 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.487692    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4jjv\" (UniqueName: \"kubernetes.io/projected/68ca634a-fcb0-4dda-8e1b-12829c81221a-kube-api-access-b4jjv\") pod \"limitador-limitador-84994f7ffc-v24tf\" (UID: \"68ca634a-fcb0-4dda-8e1b-12829c81221a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.588666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.588587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68ca634a-fcb0-4dda-8e1b-12829c81221a-config-file\") pod \"limitador-limitador-84994f7ffc-v24tf\" (UID: \"68ca634a-fcb0-4dda-8e1b-12829c81221a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.588666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.588626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4jjv\" (UniqueName: \"kubernetes.io/projected/68ca634a-fcb0-4dda-8e1b-12829c81221a-kube-api-access-b4jjv\") pod \"limitador-limitador-84994f7ffc-v24tf\" (UID: \"68ca634a-fcb0-4dda-8e1b-12829c81221a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.589228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.589204    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68ca634a-fcb0-4dda-8e1b-12829c81221a-config-file\") pod \"limitador-limitador-84994f7ffc-v24tf\" (UID: \"68ca634a-fcb0-4dda-8e1b-12829c81221a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.596536 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.596507    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4jjv\" (UniqueName: \"kubernetes.io/projected/68ca634a-fcb0-4dda-8e1b-12829c81221a-kube-api-access-b4jjv\") pod \"limitador-limitador-84994f7ffc-v24tf\" (UID: \"68ca634a-fcb0-4dda-8e1b-12829c81221a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.658007 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.657981    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:35:58.660046 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:58.660008    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47224bbe_ff14_4459_8616_4f0390fe10e2.slice/crio-fe9482b82a186e5d884ef1cb2300161f0bca34c332a6849bb26b977b8c7e33e0 WatchSource:0}: Error finding container fe9482b82a186e5d884ef1cb2300161f0bca34c332a6849bb26b977b8c7e33e0: Status 404 returned error can't find the container with id fe9482b82a186e5d884ef1cb2300161f0bca34c332a6849bb26b977b8c7e33e0
Sep 10 14:35:58.672267 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.672238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v" event={"ID":"47224bbe-ff14-4459-8616-4f0390fe10e2","Type":"ContainerStarted","Data":"fe9482b82a186e5d884ef1cb2300161f0bca34c332a6849bb26b977b8c7e33e0"}
Sep 10 14:35:58.761181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.761145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:58.917550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:58.917463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:59.051936 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.051908    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:35:59.053174 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:59.053148    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ca634a_fcb0_4dda_8e1b_12829c81221a.slice/crio-536d7c94b23f071684553d3fd70eaa366911c570d7f7137cf0ae9ae287255bac WatchSource:0}: Error finding container 536d7c94b23f071684553d3fd70eaa366911c570d7f7137cf0ae9ae287255bac: Status 404 returned error can't find the container with id 536d7c94b23f071684553d3fd70eaa366911c570d7f7137cf0ae9ae287255bac
Sep 10 14:35:59.099672 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.099637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:59.163946 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.163904    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-sn988"]
Sep 10 14:35:59.167648 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.167581    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:35:59.172188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.172163    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vzdhs\""
Sep 10 14:35:59.186753 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.186729    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-sn988"]
Sep 10 14:35:59.294772 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.294730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s29b2\" (UniqueName: \"kubernetes.io/projected/61402e78-2a03-4d8a-a62c-fded4ab34de7-kube-api-access-s29b2\") pod \"authorino-59c974bd48-sn988\" (UID: \"61402e78-2a03-4d8a-a62c-fded4ab34de7\") " pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:35:59.395975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.395932    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s29b2\" (UniqueName: \"kubernetes.io/projected/61402e78-2a03-4d8a-a62c-fded4ab34de7-kube-api-access-s29b2\") pod \"authorino-59c974bd48-sn988\" (UID: \"61402e78-2a03-4d8a-a62c-fded4ab34de7\") " pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:35:59.406569 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.406533    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s29b2\" (UniqueName: \"kubernetes.io/projected/61402e78-2a03-4d8a-a62c-fded4ab34de7-kube-api-access-s29b2\") pod \"authorino-59c974bd48-sn988\" (UID: \"61402e78-2a03-4d8a-a62c-fded4ab34de7\") " pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:35:59.478348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.478307    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:35:59.679298 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.679252    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf" event={"ID":"68ca634a-fcb0-4dda-8e1b-12829c81221a","Type":"ContainerStarted","Data":"fc347ab9a71dacfd92cf12030bd8a991cf9f72adc7f26b1c45b83d41a35671d7"}
Sep 10 14:35:59.679479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.679344    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:35:59.679479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.679375    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf" event={"ID":"68ca634a-fcb0-4dda-8e1b-12829c81221a","Type":"ContainerStarted","Data":"536d7c94b23f071684553d3fd70eaa366911c570d7f7137cf0ae9ae287255bac"}
Sep 10 14:35:59.680934 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.680906    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v" event={"ID":"47224bbe-ff14-4459-8616-4f0390fe10e2","Type":"ContainerStarted","Data":"767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce"}
Sep 10 14:35:59.681055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.680962    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:35:59.695540 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.695487    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf" podStartSLOduration=1.154248661 podStartE2EDuration="1.695469654s" totalImagesPullingTime="541.220993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:59.694692475 +0000 UTC m=+489.050505393" watchObservedRunningTime="2026-09-10 14:35:59.695469654 +0000 UTC m=+489.051282560"
Sep 10 14:35:59.711333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.711225    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v" podStartSLOduration=0.881278186 podStartE2EDuration="1.711203618s" totalImagesPullingTime="829.925432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:59.710721122 +0000 UTC m=+489.066534048" watchObservedRunningTime="2026-09-10 14:35:59.711203618 +0000 UTC m=+489.067016526"
Sep 10 14:35:59.753107 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:35:59.753078    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-sn988"]
Sep 10 14:35:59.754925 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:35:59.754897    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61402e78_2a03_4d8a_a62c_fded4ab34de7.slice/crio-1486fdd400e5415fdceadaf2d44308531dd196b8e3d1a0d9a279d64f92fd6561 WatchSource:0}: Error finding container 1486fdd400e5415fdceadaf2d44308531dd196b8e3d1a0d9a279d64f92fd6561: Status 404 returned error can't find the container with id 1486fdd400e5415fdceadaf2d44308531dd196b8e3d1a0d9a279d64f92fd6561
Sep 10 14:36:00.688051 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:00.688011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-sn988" event={"ID":"61402e78-2a03-4d8a-a62c-fded4ab34de7","Type":"ContainerStarted","Data":"1486fdd400e5415fdceadaf2d44308531dd196b8e3d1a0d9a279d64f92fd6561"}
Sep 10 14:36:01.694537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:01.694501    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-sn988" event={"ID":"61402e78-2a03-4d8a-a62c-fded4ab34de7","Type":"ContainerStarted","Data":"0cfcfd1cb32cc5574338e1f6549bd673c23eb3de1ebcdb926e98da2c867c2e4c"}
Sep 10 14:36:01.720627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:01.720575    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-sn988" podStartSLOduration=1.20263354 podStartE2EDuration="2.720558252s" totalImagesPullingTime="1.517924712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:01.718881845 +0000 UTC m=+491.074694751" watchObservedRunningTime="2026-09-10 14:36:01.720558252 +0000 UTC m=+491.076371206"
Sep 10 14:36:10.688986 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:10.688949    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-v24tf"
Sep 10 14:36:10.689460 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:10.689201    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:36:10.746901 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:10.746868    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:36:10.747122 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:10.747076    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v" podUID="47224bbe-ff14-4459-8616-4f0390fe10e2" containerName="limitador" containerID="cri-o://767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce" gracePeriod=30
Sep 10 14:36:11.682508 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.682484    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:36:11.744783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.744748    2560 generic.go:356] "Generic (PLEG): container finished" podID="47224bbe-ff14-4459-8616-4f0390fe10e2" containerID="767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce" exitCode=0
Sep 10 14:36:11.745247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.744788    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v"
Sep 10 14:36:11.745247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.744827    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v" event={"ID":"47224bbe-ff14-4459-8616-4f0390fe10e2","Type":"ContainerDied","Data":"767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce"}
Sep 10 14:36:11.745247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.744875    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cqz2v" event={"ID":"47224bbe-ff14-4459-8616-4f0390fe10e2","Type":"ContainerDied","Data":"fe9482b82a186e5d884ef1cb2300161f0bca34c332a6849bb26b977b8c7e33e0"}
Sep 10 14:36:11.745247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.744897    2560 scope.go:117] "RemoveContainer" containerID="767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce"
Sep 10 14:36:11.753361 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.753191    2560 scope.go:117] "RemoveContainer" containerID="767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce"
Sep 10 14:36:11.753535 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:36:11.753516    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce\": container with ID starting with 767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce not found: ID does not exist" containerID="767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce"
Sep 10 14:36:11.753571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.753550    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce"} err="failed to get container status \"767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce\": rpc error: code = NotFound desc = could not find container \"767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce\": container with ID starting with 767988ab1f86fe88bbdfcd1af7ad5744c9d26f1e43761de0adfbda7ac1cb68ce not found: ID does not exist"
Sep 10 14:36:11.797798 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.797767    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47224bbe-ff14-4459-8616-4f0390fe10e2-config-file\") pod \"47224bbe-ff14-4459-8616-4f0390fe10e2\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") "
Sep 10 14:36:11.797939 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.797891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmfng\" (UniqueName: \"kubernetes.io/projected/47224bbe-ff14-4459-8616-4f0390fe10e2-kube-api-access-nmfng\") pod \"47224bbe-ff14-4459-8616-4f0390fe10e2\" (UID: \"47224bbe-ff14-4459-8616-4f0390fe10e2\") "
Sep 10 14:36:11.798107 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.798082    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47224bbe-ff14-4459-8616-4f0390fe10e2-config-file" (OuterVolumeSpecName: "config-file") pod "47224bbe-ff14-4459-8616-4f0390fe10e2" (UID: "47224bbe-ff14-4459-8616-4f0390fe10e2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:36:11.799912 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.799887    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47224bbe-ff14-4459-8616-4f0390fe10e2-kube-api-access-nmfng" (OuterVolumeSpecName: "kube-api-access-nmfng") pod "47224bbe-ff14-4459-8616-4f0390fe10e2" (UID: "47224bbe-ff14-4459-8616-4f0390fe10e2"). InnerVolumeSpecName "kube-api-access-nmfng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:36:11.898983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.898894    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmfng\" (UniqueName: \"kubernetes.io/projected/47224bbe-ff14-4459-8616-4f0390fe10e2-kube-api-access-nmfng\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:36:11.898983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:11.898924    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47224bbe-ff14-4459-8616-4f0390fe10e2-config-file\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:36:12.065636 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:12.065603    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:36:12.068742 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:12.068717    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cqz2v"]
Sep 10 14:36:13.317899 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:13.317859    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47224bbe-ff14-4459-8616-4f0390fe10e2" path="/var/lib/kubelet/pods/47224bbe-ff14-4459-8616-4f0390fe10e2/volumes"
Sep 10 14:36:13.997125 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:13.997090    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.100372 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:14.100338    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.533613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:14.533580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:26.933274 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:26.933241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:27.056624 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:27.056593    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:29.646069 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:29.646036    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-sn988"]
Sep 10 14:36:29.646528 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:29.646243    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-sn988" podUID="61402e78-2a03-4d8a-a62c-fded4ab34de7" containerName="authorino" containerID="cri-o://0cfcfd1cb32cc5574338e1f6549bd673c23eb3de1ebcdb926e98da2c867c2e4c" gracePeriod=30
Sep 10 14:36:29.838494 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:29.838457    2560 generic.go:356] "Generic (PLEG): container finished" podID="61402e78-2a03-4d8a-a62c-fded4ab34de7" containerID="0cfcfd1cb32cc5574338e1f6549bd673c23eb3de1ebcdb926e98da2c867c2e4c" exitCode=0
Sep 10 14:36:29.838641 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:29.838537    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-sn988" event={"ID":"61402e78-2a03-4d8a-a62c-fded4ab34de7","Type":"ContainerDied","Data":"0cfcfd1cb32cc5574338e1f6549bd673c23eb3de1ebcdb926e98da2c867c2e4c"}
Sep 10 14:36:29.891063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:29.891039    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:36:30.048658 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.048626    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s29b2\" (UniqueName: \"kubernetes.io/projected/61402e78-2a03-4d8a-a62c-fded4ab34de7-kube-api-access-s29b2\") pod \"61402e78-2a03-4d8a-a62c-fded4ab34de7\" (UID: \"61402e78-2a03-4d8a-a62c-fded4ab34de7\") "
Sep 10 14:36:30.050694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.050672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61402e78-2a03-4d8a-a62c-fded4ab34de7-kube-api-access-s29b2" (OuterVolumeSpecName: "kube-api-access-s29b2") pod "61402e78-2a03-4d8a-a62c-fded4ab34de7" (UID: "61402e78-2a03-4d8a-a62c-fded4ab34de7"). InnerVolumeSpecName "kube-api-access-s29b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:36:30.149456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.149422    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s29b2\" (UniqueName: \"kubernetes.io/projected/61402e78-2a03-4d8a-a62c-fded4ab34de7-kube-api-access-s29b2\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:36:30.844698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.844664    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-sn988" event={"ID":"61402e78-2a03-4d8a-a62c-fded4ab34de7","Type":"ContainerDied","Data":"1486fdd400e5415fdceadaf2d44308531dd196b8e3d1a0d9a279d64f92fd6561"}
Sep 10 14:36:30.844698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.844685    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-sn988"
Sep 10 14:36:30.845186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.844711    2560 scope.go:117] "RemoveContainer" containerID="0cfcfd1cb32cc5574338e1f6549bd673c23eb3de1ebcdb926e98da2c867c2e4c"
Sep 10 14:36:30.864236 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.864211    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-sn988"]
Sep 10 14:36:30.867589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:30.867567    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-sn988"]
Sep 10 14:36:31.320494 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:31.320461    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61402e78-2a03-4d8a-a62c-fded4ab34de7" path="/var/lib/kubelet/pods/61402e78-2a03-4d8a-a62c-fded4ab34de7/volumes"
Sep 10 14:36:34.418620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.418577    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:34.444150 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444114    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6654675b9d-f6msj"]
Sep 10 14:36:34.444466 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444452    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61402e78-2a03-4d8a-a62c-fded4ab34de7" containerName="authorino"
Sep 10 14:36:34.444523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444468    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="61402e78-2a03-4d8a-a62c-fded4ab34de7" containerName="authorino"
Sep 10 14:36:34.444523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444490    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47224bbe-ff14-4459-8616-4f0390fe10e2" containerName="limitador"
Sep 10 14:36:34.444523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444496    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47224bbe-ff14-4459-8616-4f0390fe10e2" containerName="limitador"
Sep 10 14:36:34.444632 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444548    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="61402e78-2a03-4d8a-a62c-fded4ab34de7" containerName="authorino"
Sep 10 14:36:34.444632 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.444556    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47224bbe-ff14-4459-8616-4f0390fe10e2" containerName="limitador"
Sep 10 14:36:34.449556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.449529    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.451447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.451426    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:36:34.451605 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.451456    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:36:34.452187 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.452169    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6hmtm\""
Sep 10 14:36:34.452301 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.452201    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:36:34.456436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.456417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6654675b9d-f6msj"]
Sep 10 14:36:34.514209 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.514183    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:34.566535 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.566493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:34.585537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.585514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-webhook-cert\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.585646 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.585546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bgzn\" (UniqueName: \"kubernetes.io/projected/ddc684a9-daeb-4908-b18c-1e380912b796-kube-api-access-7bgzn\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.585646 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.585569    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-metrics-certs\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.623606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.623573    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-srtf8"]
Sep 10 14:36:34.627271 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.627249    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.635493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.635469    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-srtf8"]
Sep 10 14:36:34.683792 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.683724    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:34.686926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.686907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-webhook-cert\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.686983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.686940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bgzn\" (UniqueName: \"kubernetes.io/projected/ddc684a9-daeb-4908-b18c-1e380912b796-kube-api-access-7bgzn\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.686983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.686962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-metrics-certs\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.689296 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.689267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-webhook-cert\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.689408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.689341    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-metrics-certs\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.694655 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.694633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bgzn\" (UniqueName: \"kubernetes.io/projected/ddc684a9-daeb-4908-b18c-1e380912b796-kube-api-access-7bgzn\") pod \"maas-controller-6654675b9d-f6msj\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") " pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.761458 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.761421    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:34.788146 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.788117    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-webhook-cert\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.788275 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.788162    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-metrics-certs\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.788275 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.788202    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm24r\" (UniqueName: \"kubernetes.io/projected/6b5488a8-6dc5-436e-a86f-72c3ee732e06-kube-api-access-sm24r\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.889545 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.889508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-webhook-cert\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.889721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.889554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-metrics-certs\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.889721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.889581    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm24r\" (UniqueName: \"kubernetes.io/projected/6b5488a8-6dc5-436e-a86f-72c3ee732e06-kube-api-access-sm24r\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.892134 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.892101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-webhook-cert\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.892262 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.892153    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-metrics-certs\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.896538 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.896517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm24r\" (UniqueName: \"kubernetes.io/projected/6b5488a8-6dc5-436e-a86f-72c3ee732e06-kube-api-access-sm24r\") pod \"maas-controller-7b457c4dbd-srtf8\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") " pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:34.938439 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:34.938410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:35.062506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:35.062481    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6654675b9d-f6msj"]
Sep 10 14:36:35.064394 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:36:35.064363    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc684a9_daeb_4908_b18c_1e380912b796.slice/crio-763f50401a680a8fcdf7e03346b9f21c074b6768c9acf2ad6d431564e41f28b1 WatchSource:0}: Error finding container 763f50401a680a8fcdf7e03346b9f21c074b6768c9acf2ad6d431564e41f28b1: Status 404 returned error can't find the container with id 763f50401a680a8fcdf7e03346b9f21c074b6768c9acf2ad6d431564e41f28b1
Sep 10 14:36:35.210166 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:35.210143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-srtf8"]
Sep 10 14:36:35.211826 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:36:35.211804    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5488a8_6dc5_436e_a86f_72c3ee732e06.slice/crio-001ebf17fb99bf3671d49346144e3214b24e1cebb731720d7aa223b9ca0a9b4f WatchSource:0}: Error finding container 001ebf17fb99bf3671d49346144e3214b24e1cebb731720d7aa223b9ca0a9b4f: Status 404 returned error can't find the container with id 001ebf17fb99bf3671d49346144e3214b24e1cebb731720d7aa223b9ca0a9b4f
Sep 10 14:36:35.874529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:35.874480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6654675b9d-f6msj" event={"ID":"ddc684a9-daeb-4908-b18c-1e380912b796","Type":"ContainerStarted","Data":"763f50401a680a8fcdf7e03346b9f21c074b6768c9acf2ad6d431564e41f28b1"}
Sep 10 14:36:35.876075 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:35.876045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-srtf8" event={"ID":"6b5488a8-6dc5-436e-a86f-72c3ee732e06","Type":"ContainerStarted","Data":"001ebf17fb99bf3671d49346144e3214b24e1cebb731720d7aa223b9ca0a9b4f"}
Sep 10 14:36:37.869490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.869458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:37.889080 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.889036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-srtf8" event={"ID":"6b5488a8-6dc5-436e-a86f-72c3ee732e06","Type":"ContainerStarted","Data":"53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14"}
Sep 10 14:36:37.889256 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.889100    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:37.890649 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.890620    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6654675b9d-f6msj" event={"ID":"ddc684a9-daeb-4908-b18c-1e380912b796","Type":"ContainerStarted","Data":"36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8"}
Sep 10 14:36:37.890759 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.890741    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:37.906990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.906944    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b457c4dbd-srtf8" podStartSLOduration=1.429501849 podStartE2EDuration="3.90692898s" totalImagesPullingTime="2.477427131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:37.904930105 +0000 UTC m=+527.260743013" watchObservedRunningTime="2026-09-10 14:36:37.90692898 +0000 UTC m=+527.262741887"
Sep 10 14:36:37.925179 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.925129    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6654675b9d-f6msj" podStartSLOduration=1.2933487129999999 podStartE2EDuration="3.925113127s" totalImagesPullingTime="2.631764414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:37.923410062 +0000 UTC m=+527.279223013" watchObservedRunningTime="2026-09-10 14:36:37.925113127 +0000 UTC m=+527.280926051"
Sep 10 14:36:37.974746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:37.974719    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:48.897912 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:48.897838    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:48.898259 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:48.898018    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 14:36:48.963851 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:48.963814    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6654675b9d-f6msj"]
Sep 10 14:36:48.964099 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:48.964034    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6654675b9d-f6msj" podUID="ddc684a9-daeb-4908-b18c-1e380912b796" containerName="manager" containerID="cri-o://36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8" gracePeriod=30
Sep 10 14:36:49.310518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.310488    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:49.311463 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.311443    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-webhook-cert\") pod \"ddc684a9-daeb-4908-b18c-1e380912b796\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") "
Sep 10 14:36:49.311532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.311473    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-metrics-certs\") pod \"ddc684a9-daeb-4908-b18c-1e380912b796\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") "
Sep 10 14:36:49.311594 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.311545    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bgzn\" (UniqueName: \"kubernetes.io/projected/ddc684a9-daeb-4908-b18c-1e380912b796-kube-api-access-7bgzn\") pod \"ddc684a9-daeb-4908-b18c-1e380912b796\" (UID: \"ddc684a9-daeb-4908-b18c-1e380912b796\") "
Sep 10 14:36:49.313728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.313698    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ddc684a9-daeb-4908-b18c-1e380912b796" (UID: "ddc684a9-daeb-4908-b18c-1e380912b796"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:36:49.313830 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.313732    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ddc684a9-daeb-4908-b18c-1e380912b796" (UID: "ddc684a9-daeb-4908-b18c-1e380912b796"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:36:49.313888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.313828    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc684a9-daeb-4908-b18c-1e380912b796-kube-api-access-7bgzn" (OuterVolumeSpecName: "kube-api-access-7bgzn") pod "ddc684a9-daeb-4908-b18c-1e380912b796" (UID: "ddc684a9-daeb-4908-b18c-1e380912b796"). InnerVolumeSpecName "kube-api-access-7bgzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:36:49.412865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.412830    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bgzn\" (UniqueName: \"kubernetes.io/projected/ddc684a9-daeb-4908-b18c-1e380912b796-kube-api-access-7bgzn\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:36:49.412865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.412863    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-webhook-cert\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:36:49.412865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.412875    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc684a9-daeb-4908-b18c-1e380912b796-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:36:49.954213 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.954172    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddc684a9-daeb-4908-b18c-1e380912b796" containerID="36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8" exitCode=0
Sep 10 14:36:49.954738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.954240    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6654675b9d-f6msj"
Sep 10 14:36:49.954738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.954255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6654675b9d-f6msj" event={"ID":"ddc684a9-daeb-4908-b18c-1e380912b796","Type":"ContainerDied","Data":"36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8"}
Sep 10 14:36:49.954738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.954353    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6654675b9d-f6msj" event={"ID":"ddc684a9-daeb-4908-b18c-1e380912b796","Type":"ContainerDied","Data":"763f50401a680a8fcdf7e03346b9f21c074b6768c9acf2ad6d431564e41f28b1"}
Sep 10 14:36:49.954738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.954377    2560 scope.go:117] "RemoveContainer" containerID="36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8"
Sep 10 14:36:49.963428 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.963411    2560 scope.go:117] "RemoveContainer" containerID="36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8"
Sep 10 14:36:49.963684 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:36:49.963666    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8\": container with ID starting with 36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8 not found: ID does not exist" containerID="36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8"
Sep 10 14:36:49.963741 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.963691    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8"} err="failed to get container status \"36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8\": rpc error: code = NotFound desc = could not find container \"36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8\": container with ID starting with 36494ee7aa28ccd6bdc22f9fc18673ea8c26917310ca48952268114fa017cdb8 not found: ID does not exist"
Sep 10 14:36:49.970304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.970268    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6654675b9d-f6msj"]
Sep 10 14:36:49.975538 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:49.975518    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6654675b9d-f6msj"]
Sep 10 14:36:51.317585 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:36:51.317547    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc684a9-daeb-4908-b18c-1e380912b796" path="/var/lib/kubelet/pods/ddc684a9-daeb-4908-b18c-1e380912b796/volumes"
Sep 10 14:37:11.715597 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:11.715559    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:11.808255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:11.808219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.132051 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.132010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.233787 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.233754    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"]
Sep 10 14:37:13.234090 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.234077    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc684a9-daeb-4908-b18c-1e380912b796" containerName="manager"
Sep 10 14:37:13.234169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.234090    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc684a9-daeb-4908-b18c-1e380912b796" containerName="manager"
Sep 10 14:37:13.234169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.234161    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc684a9-daeb-4908-b18c-1e380912b796" containerName="manager"
Sep 10 14:37:13.238304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.238255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.242420 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.242397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.244545 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.244522    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:37:13.244667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.244597    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 14:37:13.245343 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.245323    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:37:13.245470 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.245345    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:37:13.245675 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.245348    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7j4nl\""
Sep 10 14:37:13.251891 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.251479    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"]
Sep 10 14:37:13.298427 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.298388    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g729k\" (UniqueName: \"kubernetes.io/projected/91f60dbc-7556-4794-96dc-8148f888e5d3-kube-api-access-g729k\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.298613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.298437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.298613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.298490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f60dbc-7556-4794-96dc-8148f888e5d3-openshift-service-ca\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.298613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.298511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.399404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.399309    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f60dbc-7556-4794-96dc-8148f888e5d3-openshift-service-ca\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.399404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.399354    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.399626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.399429    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g729k\" (UniqueName: \"kubernetes.io/projected/91f60dbc-7556-4794-96dc-8148f888e5d3-kube-api-access-g729k\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.399626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.399457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.399626 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:37:13.399471    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 14:37:13.399626 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:37:13.399558    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs podName:91f60dbc-7556-4794-96dc-8148f888e5d3 nodeName:}" failed. No retries permitted until 2026-09-10 14:37:13.899538212 +0000 UTC m=+563.255351111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs") pod "maas-api-59bbbd559c-pzk5x" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3") : secret "maas-api-metrics-tls" not found
Sep 10 14:37:13.399626 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:37:13.399569    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 14:37:13.399825 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:37:13.399636    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls podName:91f60dbc-7556-4794-96dc-8148f888e5d3 nodeName:}" failed. No retries permitted until 2026-09-10 14:37:13.899617015 +0000 UTC m=+563.255429907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls") pod "maas-api-59bbbd559c-pzk5x" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3") : secret "maas-api-serving-cert" not found
Sep 10 14:37:13.400122 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.400101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f60dbc-7556-4794-96dc-8148f888e5d3-openshift-service-ca\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.410452 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.410426    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g729k\" (UniqueName: \"kubernetes.io/projected/91f60dbc-7556-4794-96dc-8148f888e5d3-kube-api-access-g729k\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.628975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.628947    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.725819 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.725788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.772814 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.772779    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-bzddb"]
Sep 10 14:37:13.777045 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.777026    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:13.779189 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.779165    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-z6jmg\""
Sep 10 14:37:13.779325 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.779205    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 14:37:13.786305 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.786249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-bzddb"]
Sep 10 14:37:13.804479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.804454    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.808535 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.808509    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"]
Sep 10 14:37:13.813014 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.812991    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:37:13.813144 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.813094    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:13.822723 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.822697    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"]
Sep 10 14:37:13.904797 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.904753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.904984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.904906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbq8j\" (UniqueName: \"kubernetes.io/projected/427792ba-b7e1-4f48-957c-feec063e4e34-kube-api-access-sbq8j\") pod \"payload-processing-5b54b7fbb7-bzddb\" (UID: \"427792ba-b7e1-4f48-957c-feec063e4e34\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:13.904984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.904944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z4k2\" (UniqueName: \"kubernetes.io/projected/9e1afda4-6e02-4a5b-89e0-206473f32957-kube-api-access-9z4k2\") pod \"payload-pre-processing-59fbc47bf8-lgfb6\" (UID: \"9e1afda4-6e02-4a5b-89e0-206473f32957\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:13.905098 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.904995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:13.905098 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.905048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/427792ba-b7e1-4f48-957c-feec063e4e34-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-bzddb\" (UID: \"427792ba-b7e1-4f48-957c-feec063e4e34\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:13.905098 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.905087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1afda4-6e02-4a5b-89e0-206473f32957-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lgfb6\" (UID: \"9e1afda4-6e02-4a5b-89e0-206473f32957\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:13.906726 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:37:13.906700    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 14:37:13.906945 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:37:13.906930    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls podName:91f60dbc-7556-4794-96dc-8148f888e5d3 nodeName:}" failed. No retries permitted until 2026-09-10 14:37:14.906903499 +0000 UTC m=+564.262716398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls") pod "maas-api-59bbbd559c-pzk5x" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3") : secret "maas-api-serving-cert" not found
Sep 10 14:37:13.913224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:13.913193    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:14.005843 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.005743    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbq8j\" (UniqueName: \"kubernetes.io/projected/427792ba-b7e1-4f48-957c-feec063e4e34-kube-api-access-sbq8j\") pod \"payload-processing-5b54b7fbb7-bzddb\" (UID: \"427792ba-b7e1-4f48-957c-feec063e4e34\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:14.005843 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.005800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z4k2\" (UniqueName: \"kubernetes.io/projected/9e1afda4-6e02-4a5b-89e0-206473f32957-kube-api-access-9z4k2\") pod \"payload-pre-processing-59fbc47bf8-lgfb6\" (UID: \"9e1afda4-6e02-4a5b-89e0-206473f32957\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:14.006057 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.005872    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/427792ba-b7e1-4f48-957c-feec063e4e34-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-bzddb\" (UID: \"427792ba-b7e1-4f48-957c-feec063e4e34\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:14.006057 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.005902    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1afda4-6e02-4a5b-89e0-206473f32957-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lgfb6\" (UID: \"9e1afda4-6e02-4a5b-89e0-206473f32957\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:14.006841 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.006816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/427792ba-b7e1-4f48-957c-feec063e4e34-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-bzddb\" (UID: \"427792ba-b7e1-4f48-957c-feec063e4e34\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:14.006897 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.006816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1afda4-6e02-4a5b-89e0-206473f32957-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lgfb6\" (UID: \"9e1afda4-6e02-4a5b-89e0-206473f32957\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:14.013354 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.013331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z4k2\" (UniqueName: \"kubernetes.io/projected/9e1afda4-6e02-4a5b-89e0-206473f32957-kube-api-access-9z4k2\") pod \"payload-pre-processing-59fbc47bf8-lgfb6\" (UID: \"9e1afda4-6e02-4a5b-89e0-206473f32957\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:14.013468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.013374    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbq8j\" (UniqueName: \"kubernetes.io/projected/427792ba-b7e1-4f48-957c-feec063e4e34-kube-api-access-sbq8j\") pod \"payload-processing-5b54b7fbb7-bzddb\" (UID: \"427792ba-b7e1-4f48-957c-feec063e4e34\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:14.088693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.088659    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:14.109450 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.109413    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:14.124519 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.124494    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:14.365188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.365164    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-bzddb"]
Sep 10 14:37:14.366766 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:37:14.366743    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427792ba_b7e1_4f48_957c_feec063e4e34.slice/crio-8119a564f795f9187070fc02a9a95d9cad338b96103f1558772d0a89fc0ac040 WatchSource:0}: Error finding container 8119a564f795f9187070fc02a9a95d9cad338b96103f1558772d0a89fc0ac040: Status 404 returned error can't find the container with id 8119a564f795f9187070fc02a9a95d9cad338b96103f1558772d0a89fc0ac040
Sep 10 14:37:14.404822 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.404795    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"]
Sep 10 14:37:14.406188 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:37:14.406163    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1afda4_6e02_4a5b_89e0_206473f32957.slice/crio-352fcafc7a66df7d79629e056893848784f20137e94cdfc8ed4843424ddd366d WatchSource:0}: Error finding container 352fcafc7a66df7d79629e056893848784f20137e94cdfc8ed4843424ddd366d: Status 404 returned error can't find the container with id 352fcafc7a66df7d79629e056893848784f20137e94cdfc8ed4843424ddd366d
Sep 10 14:37:14.914538 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.914497    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:14.933665 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:14.933632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") pod \"maas-api-59bbbd559c-pzk5x\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") " pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:15.056895 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:15.056841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:15.090191 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:15.090152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6" event={"ID":"9e1afda4-6e02-4a5b-89e0-206473f32957","Type":"ContainerStarted","Data":"352fcafc7a66df7d79629e056893848784f20137e94cdfc8ed4843424ddd366d"}
Sep 10 14:37:15.091840 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:15.091783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb" event={"ID":"427792ba-b7e1-4f48-957c-feec063e4e34","Type":"ContainerStarted","Data":"8119a564f795f9187070fc02a9a95d9cad338b96103f1558772d0a89fc0ac040"}
Sep 10 14:37:15.379634 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:15.379143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"]
Sep 10 14:37:15.391014 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:37:15.388927    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f60dbc_7556_4794_96dc_8148f888e5d3.slice/crio-8d23142b7c36ec4516d183c150bac51426c0316bc281908e54d3c7fc9b956216 WatchSource:0}: Error finding container 8d23142b7c36ec4516d183c150bac51426c0316bc281908e54d3c7fc9b956216: Status 404 returned error can't find the container with id 8d23142b7c36ec4516d183c150bac51426c0316bc281908e54d3c7fc9b956216
Sep 10 14:37:16.097635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:16.097598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x" event={"ID":"91f60dbc-7556-4794-96dc-8148f888e5d3","Type":"ContainerStarted","Data":"8d23142b7c36ec4516d183c150bac51426c0316bc281908e54d3c7fc9b956216"}
Sep 10 14:37:17.106697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:17.106623    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb" event={"ID":"427792ba-b7e1-4f48-957c-feec063e4e34","Type":"ContainerStarted","Data":"4f99a86d268890c97d5b4c11b3c884d50b1303b21c1b6648e023740c99a01d5d"}
Sep 10 14:37:17.107169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:17.106786    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:17.108715 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:17.108661    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6" event={"ID":"9e1afda4-6e02-4a5b-89e0-206473f32957","Type":"ContainerStarted","Data":"3fe066765feba5d849f7dad79f3ff01f3520b582acf5a52c63d6fa83cb7f0c78"}
Sep 10 14:37:17.108919 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:17.108888    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:17.124746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:17.124696    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb" podStartSLOduration=2.335255364 podStartE2EDuration="4.124676907s" totalImagesPullingTime="1.789421543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:17.12220561 +0000 UTC m=+566.478018536" watchObservedRunningTime="2026-09-10 14:37:17.124676907 +0000 UTC m=+566.480489814"
Sep 10 14:37:17.138774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:17.138723    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6" podStartSLOduration=2.39538607 podStartE2EDuration="4.138709233s" totalImagesPullingTime="1.743323163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:17.136339754 +0000 UTC m=+566.492152684" watchObservedRunningTime="2026-09-10 14:37:17.138709233 +0000 UTC m=+566.494522139"
Sep 10 14:37:18.116112 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:18.116069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x" event={"ID":"91f60dbc-7556-4794-96dc-8148f888e5d3","Type":"ContainerStarted","Data":"9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3"}
Sep 10 14:37:18.133156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:18.133111    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x" podStartSLOduration=3.152941732 podStartE2EDuration="5.133095684s" totalImagesPullingTime="1.980153952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:18.132309124 +0000 UTC m=+567.488122052" watchObservedRunningTime="2026-09-10 14:37:18.133095684 +0000 UTC m=+567.488908592"
Sep 10 14:37:19.121388 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:19.121360    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:25.131881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:25.131849    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 14:37:26.119000 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:26.118963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:26.216790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:26.216762    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:28.117947 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:28.117918    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-bzddb"
Sep 10 14:37:28.118342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:28.117985    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lgfb6"
Sep 10 14:37:32.465619 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:32.465588    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:35.418477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:35.418446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:39.721211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:39.721178    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:40.085254 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:40.085173    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:37:55.814029 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.813992    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"]
Sep 10 14:37:55.818846 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.818830    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:55.821057 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.821037    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 14:37:55.821185 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.821085    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tw2h4\""
Sep 10 14:37:55.821185 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.821085    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 14:37:55.821699 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.821682    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 14:37:55.826231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.826210    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"]
Sep 10 14:37:55.854469 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.854438    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:55.957155 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.957114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:55.957363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.957174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqz8z\" (UniqueName: \"kubernetes.io/projected/eda000be-a4db-46a6-a7aa-45eacdb38efe-kube-api-access-pqz8z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:55.957363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.957241    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:55.957363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.957301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda000be-a4db-46a6-a7aa-45eacdb38efe-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:55.957363 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.957338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:55.957531 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:55.957383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.058597 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.058557    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqz8z\" (UniqueName: \"kubernetes.io/projected/eda000be-a4db-46a6-a7aa-45eacdb38efe-kube-api-access-pqz8z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.058597 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.058601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.058848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.058637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda000be-a4db-46a6-a7aa-45eacdb38efe-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.058848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.058677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.058848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.058702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.058848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.058776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.059188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.059151    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.059188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.059183    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.059396 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.059221    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.061074 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.061038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda000be-a4db-46a6-a7aa-45eacdb38efe-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.061186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.061173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda000be-a4db-46a6-a7aa-45eacdb38efe-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.065813 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.065756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqz8z\" (UniqueName: \"kubernetes.io/projected/eda000be-a4db-46a6-a7aa-45eacdb38efe-kube-api-access-pqz8z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ktfz5\" (UID: \"eda000be-a4db-46a6-a7aa-45eacdb38efe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.130041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.130001    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:37:56.404751 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.404714    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"]
Sep 10 14:37:56.407562 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:37:56.407538    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda000be_a4db_46a6_a7aa_45eacdb38efe.slice/crio-f90887567e7de0331405d62216ea1a9bfd1b56bcb3d081fc177382e333cc04ca WatchSource:0}: Error finding container f90887567e7de0331405d62216ea1a9bfd1b56bcb3d081fc177382e333cc04ca: Status 404 returned error can't find the container with id f90887567e7de0331405d62216ea1a9bfd1b56bcb3d081fc177382e333cc04ca
Sep 10 14:37:56.408823 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.408803    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:37:56.490934 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:56.490895    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:37:57.327916 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:57.327868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5" event={"ID":"eda000be-a4db-46a6-a7aa-45eacdb38efe","Type":"ContainerStarted","Data":"f90887567e7de0331405d62216ea1a9bfd1b56bcb3d081fc177382e333cc04ca"}
Sep 10 14:37:58.334565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:58.334523    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5" event={"ID":"eda000be-a4db-46a6-a7aa-45eacdb38efe","Type":"ContainerStarted","Data":"23a404a97d196570f0e830db2faa24f791317d89302a9fa141d4ade441e953be"}
Sep 10 14:37:58.355639 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:37:58.355584    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5" podStartSLOduration=2.342295541 podStartE2EDuration="3.355569289s" totalImagesPullingTime="1.013273748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:58.353199244 +0000 UTC m=+607.709012161" watchObservedRunningTime="2026-09-10 14:37:58.355569289 +0000 UTC m=+607.711382196"
Sep 10 14:38:06.130784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:06.130743    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:38:06.131220 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:06.130804    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:38:06.143591 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:06.143558    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:38:06.386953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:06.386861    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ktfz5"
Sep 10 14:38:06.630534 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:06.630502    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:08.404152 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:08.404125    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:25.668066 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:25.667984    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:25.723449 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:25.723420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:28.176391 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:28.176360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:29.074385 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:29.074301    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:38:41.393469 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:41.393432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:50.827436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.827405    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"]
Sep 10 14:38:50.830814 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.830791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:50.832940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.832920    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 14:38:50.840721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.840698    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"]
Sep 10 14:38:50.862390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.862361    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:50.922803 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.922759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:50.922973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.922821    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:50.922973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.922842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:50.922973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.922887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7j5j\" (UniqueName: \"kubernetes.io/projected/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-kube-api-access-g7j5j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:50.922973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.922949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:50.923112 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:50.923021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024424 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024389    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7j5j\" (UniqueName: \"kubernetes.io/projected/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-kube-api-access-g7j5j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024500    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024794 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024794 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.024963 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.024937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.025101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.025054    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.025101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.025068    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.027316 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.027291    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.027596 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.027577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.032563 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.032540    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7j5j\" (UniqueName: \"kubernetes.io/projected/c0a1d7a4-8f14-455a-a16c-1e4a77060d38-kube-api-access-g7j5j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8s2hr\" (UID: \"c0a1d7a4-8f14-455a-a16c-1e4a77060d38\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.143387 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.143270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:38:51.417924 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.417841    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"]
Sep 10 14:38:51.421750 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:38:51.421723    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a1d7a4_8f14_455a_a16c_1e4a77060d38.slice/crio-924e69bd0b3f44577675bbb53b45620c5a7304a5a42ce9c2547db4f06d473235 WatchSource:0}: Error finding container 924e69bd0b3f44577675bbb53b45620c5a7304a5a42ce9c2547db4f06d473235: Status 404 returned error can't find the container with id 924e69bd0b3f44577675bbb53b45620c5a7304a5a42ce9c2547db4f06d473235
Sep 10 14:38:51.619333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:51.619293    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr" event={"ID":"c0a1d7a4-8f14-455a-a16c-1e4a77060d38","Type":"ContainerStarted","Data":"924e69bd0b3f44577675bbb53b45620c5a7304a5a42ce9c2547db4f06d473235"}
Sep 10 14:38:52.075626 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:52.075590    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:38:52.625972 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:52.625936    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr" event={"ID":"c0a1d7a4-8f14-455a-a16c-1e4a77060d38","Type":"ContainerStarted","Data":"0a890dd8e048818622a2f03bba1053246e9fb4270b4018caf28a7ece95cf72fe"}
Sep 10 14:38:52.642621 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:52.642570    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr" podStartSLOduration=2.441731809 podStartE2EDuration="2.642552755s" totalImagesPullingTime="200.820946ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:38:52.641555775 +0000 UTC m=+661.997368716" watchObservedRunningTime="2026-09-10 14:38:52.642552755 +0000 UTC m=+661.998365663"
Sep 10 14:38:53.339599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:53.339564    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:57.778790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:38:57.778747    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:39:01.143616 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:01.143560    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:39:01.144040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:01.143641    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:39:01.156632 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:01.156602    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:39:01.687571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:01.687538    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8s2hr"
Sep 10 14:39:03.617448 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.617413    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"]
Sep 10 14:39:03.621837 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.621810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.624391 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.624367    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 14:39:03.628823 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.628800    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"]
Sep 10 14:39:03.657407 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.657376    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:03.731751 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.731711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f722f0-9774-4876-9f5d-5de63651a8e0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.731926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.731768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pskdx\" (UniqueName: \"kubernetes.io/projected/33f722f0-9774-4876-9f5d-5de63651a8e0-kube-api-access-pskdx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.731926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.731789    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.731926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.731873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.731926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.731910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.732091 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.731934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f722f0-9774-4876-9f5d-5de63651a8e0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833461 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pskdx\" (UniqueName: \"kubernetes.io/projected/33f722f0-9774-4876-9f5d-5de63651a8e0-kube-api-access-pskdx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833461 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833461 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833461 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833461 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.833836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833822    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.834010 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.833854    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.835795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.835771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f722f0-9774-4876-9f5d-5de63651a8e0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.835935 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.835918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f722f0-9774-4876-9f5d-5de63651a8e0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.840630 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.840610    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pskdx\" (UniqueName: \"kubernetes.io/projected/33f722f0-9774-4876-9f5d-5de63651a8e0-kube-api-access-pskdx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8db7v\" (UID: \"33f722f0-9774-4876-9f5d-5de63651a8e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:03.936087 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:03.935997    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:04.209574 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:04.209547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"]
Sep 10 14:39:04.211009 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:39:04.210983    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f722f0_9774_4876_9f5d_5de63651a8e0.slice/crio-cb06eb65af6bbb4e3ac05780da5150cdff272eb683a5cbe02fa4ca59e98db3e6 WatchSource:0}: Error finding container cb06eb65af6bbb4e3ac05780da5150cdff272eb683a5cbe02fa4ca59e98db3e6: Status 404 returned error can't find the container with id cb06eb65af6bbb4e3ac05780da5150cdff272eb683a5cbe02fa4ca59e98db3e6
Sep 10 14:39:04.690761 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:04.690723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v" event={"ID":"33f722f0-9774-4876-9f5d-5de63651a8e0","Type":"ContainerStarted","Data":"11a6ba88a0ef6952bf00be2ef45f18f39199034d4a46b110488827ec9d4ab9e9"}
Sep 10 14:39:04.691132 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:04.690774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v" event={"ID":"33f722f0-9774-4876-9f5d-5de63651a8e0","Type":"ContainerStarted","Data":"cb06eb65af6bbb4e3ac05780da5150cdff272eb683a5cbe02fa4ca59e98db3e6"}
Sep 10 14:39:04.711413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:04.711303    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v" podStartSLOduration=1.466918766 podStartE2EDuration="1.711274994s" totalImagesPullingTime="244.356228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:04.709180244 +0000 UTC m=+674.064993152" watchObservedRunningTime="2026-09-10 14:39:04.711274994 +0000 UTC m=+674.067087900"
Sep 10 14:39:05.288471 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:05.288430    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:39:13.936456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:13.936414    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:13.936897 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:13.936475    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:13.949328 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:13.949299    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:14.755814 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:14.755784    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8db7v"
Sep 10 14:39:21.836996 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.836961    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"]
Sep 10 14:39:21.840839 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.840817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.843038 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.843015    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 14:39:21.849230 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.849204    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"]
Sep 10 14:39:21.861676 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.861656    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:21.889786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.889754    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7z58\" (UniqueName: \"kubernetes.io/projected/9cad246b-1533-458f-b51e-4eb04945acd7-kube-api-access-s7z58\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.889955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.889795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.889955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.889876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cad246b-1533-458f-b51e-4eb04945acd7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.889955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.889925    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.890120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.889971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.890120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.890029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991261 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7z58\" (UniqueName: \"kubernetes.io/projected/9cad246b-1533-458f-b51e-4eb04945acd7-kube-api-access-s7z58\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991261 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991262    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991322    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cad246b-1533-458f-b51e-4eb04945acd7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991379    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991812 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.991881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.991851    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.993710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.993687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cad246b-1533-458f-b51e-4eb04945acd7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.993958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.993937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cad246b-1533-458f-b51e-4eb04945acd7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:21.998475 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:21.998451    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7z58\" (UniqueName: \"kubernetes.io/projected/9cad246b-1533-458f-b51e-4eb04945acd7-kube-api-access-s7z58\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb\" (UID: \"9cad246b-1533-458f-b51e-4eb04945acd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:22.150853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:22.150757    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:22.425031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:22.425000    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"]
Sep 10 14:39:22.426294 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:39:22.426253    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cad246b_1533_458f_b51e_4eb04945acd7.slice/crio-37efb92e7fac9aa3f21f6621c3a95dd89125fffa393d1bf6b8680834fafc874c WatchSource:0}: Error finding container 37efb92e7fac9aa3f21f6621c3a95dd89125fffa393d1bf6b8680834fafc874c: Status 404 returned error can't find the container with id 37efb92e7fac9aa3f21f6621c3a95dd89125fffa393d1bf6b8680834fafc874c
Sep 10 14:39:22.786947 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:22.786904    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb" event={"ID":"9cad246b-1533-458f-b51e-4eb04945acd7","Type":"ContainerStarted","Data":"37efb92e7fac9aa3f21f6621c3a95dd89125fffa393d1bf6b8680834fafc874c"}
Sep 10 14:39:23.799355 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:23.799316    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb" event={"ID":"9cad246b-1533-458f-b51e-4eb04945acd7","Type":"ContainerStarted","Data":"78c8ce6768dc8005b655eb5c2a8565055b2e2435ef4f746c5839fa0ad7f7db13"}
Sep 10 14:39:23.815874 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:23.815810    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb" podStartSLOduration=2.320437585 podStartE2EDuration="2.815788516s" totalImagesPullingTime="495.350931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:23.815689435 +0000 UTC m=+693.171502364" watchObservedRunningTime="2026-09-10 14:39:23.815788516 +0000 UTC m=+693.171601423"
Sep 10 14:39:24.094684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:24.094601    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:39:32.151678 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:32.151640    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:32.152119 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:32.151786    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:32.164619 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:32.164595    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:32.858915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:32.858881    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb"
Sep 10 14:39:45.444442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:45.444362    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:45.544565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:39:45.544530    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:08.425595 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:08.425559    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:08.523049 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:08.523016    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:09.570306 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:09.570252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:09.672800 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:09.672768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.230404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.230366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.329819 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.329791    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.440019 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.439976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.575273 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.575155    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.664245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.664181    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.762197 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.762165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.863931 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.863839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:25.965697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:25.965660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:26.361115 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:26.361077    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:26.749538 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:26.749507    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:26.819869 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:26.819818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:27.169368 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.169246    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:27.297307 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.297265    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7830-d23a35"]
Sep 10 14:40:27.300875 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.300859    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:27.302809 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.302784    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-x6mfq\""
Sep 10 14:40:27.306751 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.306729    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7830-d23a35"]
Sep 10 14:40:27.336009 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.335979    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cbwd\" (UniqueName: \"kubernetes.io/projected/e6af8578-ff63-4af8-b762-9d565681c8b4-kube-api-access-8cbwd\") pod \"test-curl-7830-d23a35\" (UID: \"e6af8578-ff63-4af8-b762-9d565681c8b4\") " pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:27.437334 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.437302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cbwd\" (UniqueName: \"kubernetes.io/projected/e6af8578-ff63-4af8-b762-9d565681c8b4-kube-api-access-8cbwd\") pod \"test-curl-7830-d23a35\" (UID: \"e6af8578-ff63-4af8-b762-9d565681c8b4\") " pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:27.444653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.444631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cbwd\" (UniqueName: \"kubernetes.io/projected/e6af8578-ff63-4af8-b762-9d565681c8b4-kube-api-access-8cbwd\") pod \"test-curl-7830-d23a35\" (UID: \"e6af8578-ff63-4af8-b762-9d565681c8b4\") " pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:27.482190 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.482156    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:40:27.611720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.611694    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:27.884714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:27.884689    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7830-d23a35"]
Sep 10 14:40:27.887173 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:27.887147    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6af8578_ff63_4af8_b762_9d565681c8b4.slice/crio-27b06abf10b8288e34adc36ca8aba075160132cee1e8eb8dca0c8a847fa94169 WatchSource:0}: Error finding container 27b06abf10b8288e34adc36ca8aba075160132cee1e8eb8dca0c8a847fa94169: Status 404 returned error can't find the container with id 27b06abf10b8288e34adc36ca8aba075160132cee1e8eb8dca0c8a847fa94169
Sep 10 14:40:28.164849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:28.164756    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7830-d23a35" event={"ID":"e6af8578-ff63-4af8-b762-9d565681c8b4","Type":"ContainerStarted","Data":"27b06abf10b8288e34adc36ca8aba075160132cee1e8eb8dca0c8a847fa94169"}
Sep 10 14:40:28.893697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:28.893657    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:29.005619 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:29.005542    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:29.095610 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:29.095554    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:29.194652 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:29.194620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:30.031294 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.031251    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"]
Sep 10 14:40:30.036580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.036559    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.038670 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.038647    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-5b0323-serving-cert\""
Sep 10 14:40:30.042864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.042843    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"]
Sep 10 14:40:30.061631 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.061596    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cad926-6a93-4025-be67-71ff4d11ca72-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.061868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.061843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-metrics-certs\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.061971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.061895    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.061971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.061953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74zbf\" (UniqueName: \"kubernetes.io/projected/e3cad926-6a93-4025-be67-71ff4d11ca72-kube-api-access-74zbf\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.089125 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.089095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:30.163432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.163398    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cad926-6a93-4025-be67-71ff4d11ca72-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.163432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.163434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-metrics-certs\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.163650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.163488    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.163650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.163527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74zbf\" (UniqueName: \"kubernetes.io/projected/e3cad926-6a93-4025-be67-71ff4d11ca72-kube-api-access-74zbf\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.164109 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.164088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cad926-6a93-4025-be67-71ff4d11ca72-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.166136 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.166115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.166244 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.166143    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-metrics-certs\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.173969 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.173922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74zbf\" (UniqueName: \"kubernetes.io/projected/e3cad926-6a93-4025-be67-71ff4d11ca72-kube-api-access-74zbf\") pod \"maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.192148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.192125    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:30.292478 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.292403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:30.351474 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.351440    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:30.633825 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.633766    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"]
Sep 10 14:40:30.635211 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:30.635166    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cad926_6a93_4025_be67_71ff4d11ca72.slice/crio-82b1f776b3ab266dc51547dadb6f56a73d0100ca863bb75a8aa077f97706eb88 WatchSource:0}: Error finding container 82b1f776b3ab266dc51547dadb6f56a73d0100ca863bb75a8aa077f97706eb88: Status 404 returned error can't find the container with id 82b1f776b3ab266dc51547dadb6f56a73d0100ca863bb75a8aa077f97706eb88
Sep 10 14:40:30.658293 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.658255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:30.758160 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.758127    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:30.764998 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.764968    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"]
Sep 10 14:40:30.769978 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.769957    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:30.772224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.772202    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-5b0323-dockercfg-st6z9\""
Sep 10 14:40:30.772364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.772203    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-5b0323\""
Sep 10 14:40:30.780998 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.780974    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"]
Sep 10 14:40:30.872560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.872524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/874b7593-06db-4f07-8a39-cbc00ad94e30-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:30.872739 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.872595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcd58\" (UniqueName: \"kubernetes.io/projected/874b7593-06db-4f07-8a39-cbc00ad94e30-kube-api-access-vcd58\") pod \"payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:30.973183 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.973147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/874b7593-06db-4f07-8a39-cbc00ad94e30-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:30.973399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.973229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcd58\" (UniqueName: \"kubernetes.io/projected/874b7593-06db-4f07-8a39-cbc00ad94e30-kube-api-access-vcd58\") pod \"payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:30.974046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.974022    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/874b7593-06db-4f07-8a39-cbc00ad94e30-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:30.980948 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:30.980918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcd58\" (UniqueName: \"kubernetes.io/projected/874b7593-06db-4f07-8a39-cbc00ad94e30-kube-api-access-vcd58\") pod \"payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:31.030942 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.030908    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"]
Sep 10 14:40:31.035184 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.035163    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.037330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.037307    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-9b1e67-serving-cert\""
Sep 10 14:40:31.044707 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.044688    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"]
Sep 10 14:40:31.071250 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.071219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:31.074496 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.074457    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.074652 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.074607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-metrics-certs\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.074652 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.074633    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.074843 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.074677    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4sch\" (UniqueName: \"kubernetes.io/projected/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-kube-api-access-t4sch\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.082273 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.082247    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:31.171783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.171752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:31.178794 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.178765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-metrics-certs\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.179016 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.178994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.179160 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.179143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4sch\" (UniqueName: \"kubernetes.io/projected/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-kube-api-access-t4sch\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.179453 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:31.179434    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-serving-cert: secret "maas-api-e2e-worker-w0-9b1e67-serving-cert" not found
Sep 10 14:40:31.179628 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:31.179617    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls podName:170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a nodeName:}" failed. No retries permitted until 2026-09-10 14:40:31.679595636 +0000 UTC m=+761.035408532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls") pod "maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" (UID: "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a") : secret "maas-api-e2e-worker-w0-9b1e67-serving-cert" not found
Sep 10 14:40:31.179743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.179727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.180597 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.180575    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.184056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.182956    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-metrics-certs\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.188486 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.188414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442" event={"ID":"e3cad926-6a93-4025-be67-71ff4d11ca72","Type":"ContainerStarted","Data":"82b1f776b3ab266dc51547dadb6f56a73d0100ca863bb75a8aa077f97706eb88"}
Sep 10 14:40:31.189773 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.189753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4sch\" (UniqueName: \"kubernetes.io/projected/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-kube-api-access-t4sch\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.313195 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.313166    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:31.365907 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.365867    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"]
Sep 10 14:40:31.366890 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:31.366869    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod874b7593_06db_4f07_8a39_cbc00ad94e30.slice/crio-76d3e1ff1acb8f730101d481f986f202fc844890f9fe1016416f3a1640aed1a9 WatchSource:0}: Error finding container 76d3e1ff1acb8f730101d481f986f202fc844890f9fe1016416f3a1640aed1a9: Status 404 returned error can't find the container with id 76d3e1ff1acb8f730101d481f986f202fc844890f9fe1016416f3a1640aed1a9
Sep 10 14:40:31.687353 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.684107    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.687353 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.686898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.720477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.720446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:31.815690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.815659    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:31.858473 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.858444    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"]
Sep 10 14:40:31.862060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.862039    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:31.864217 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.864194    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-9b1e67\""
Sep 10 14:40:31.864336 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.864314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-9b1e67-dockercfg-xn2p5\""
Sep 10 14:40:31.877341 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.877297    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"]
Sep 10 14:40:31.949076 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.948967    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:31.987711 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.987674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpjsv\" (UniqueName: \"kubernetes.io/projected/2116c427-8cb4-4383-95ce-4386943e5546-kube-api-access-vpjsv\") pod \"payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:31.987888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:31.987720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2116c427-8cb4-4383-95ce-4386943e5546-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:32.089092 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.089058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjsv\" (UniqueName: \"kubernetes.io/projected/2116c427-8cb4-4383-95ce-4386943e5546-kube-api-access-vpjsv\") pod \"payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:32.089573 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.089102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2116c427-8cb4-4383-95ce-4386943e5546-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:32.089927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.089903    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2116c427-8cb4-4383-95ce-4386943e5546-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:32.096989 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.096961    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjsv\" (UniqueName: \"kubernetes.io/projected/2116c427-8cb4-4383-95ce-4386943e5546-kube-api-access-vpjsv\") pod \"payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:32.134540 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.134503    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"]
Sep 10 14:40:32.139977 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.139949    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.142183 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.142064    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-5121f4-serving-cert\""
Sep 10 14:40:32.149254 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.149229    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"]
Sep 10 14:40:32.171811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.171766    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:32.189837 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.189788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxkcv\" (UniqueName: \"kubernetes.io/projected/5615fc2c-e1d3-41fb-a02e-2c88713a3900-kube-api-access-xxkcv\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.190197 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.190134    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.190431 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.190412    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-metrics-certs\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.190624 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.190590    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5615fc2c-e1d3-41fb-a02e-2c88713a3900-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.200918 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.200707    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442" event={"ID":"e3cad926-6a93-4025-be67-71ff4d11ca72","Type":"ContainerStarted","Data":"9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa"}
Sep 10 14:40:32.200918 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.200817    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:32.203061 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.203031    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn" event={"ID":"874b7593-06db-4f07-8a39-cbc00ad94e30","Type":"ContainerStarted","Data":"84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232"}
Sep 10 14:40:32.203206 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.203084    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn" event={"ID":"874b7593-06db-4f07-8a39-cbc00ad94e30","Type":"ContainerStarted","Data":"76d3e1ff1acb8f730101d481f986f202fc844890f9fe1016416f3a1640aed1a9"}
Sep 10 14:40:32.203318 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.203273    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:32.221615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.221546    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442" podStartSLOduration=1.6505476159999999 podStartE2EDuration="2.221508067s" totalImagesPullingTime="570.960451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:32.218617763 +0000 UTC m=+761.574430671" watchObservedRunningTime="2026-09-10 14:40:32.221508067 +0000 UTC m=+761.577320973"
Sep 10 14:40:32.222190 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.222129    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:32.232694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.232674    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"]
Sep 10 14:40:32.246431 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.244568    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn" podStartSLOduration=2.244546898 podStartE2EDuration="2.244546898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:32.2415328 +0000 UTC m=+761.597345708" watchObservedRunningTime="2026-09-10 14:40:32.244546898 +0000 UTC m=+761.600359800"
Sep 10 14:40:32.273392 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.273334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:32.291394 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.291360    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.291571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.291432    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-metrics-certs\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.291571 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.291470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5615fc2c-e1d3-41fb-a02e-2c88713a3900-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.291693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.291594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxkcv\" (UniqueName: \"kubernetes.io/projected/5615fc2c-e1d3-41fb-a02e-2c88713a3900-kube-api-access-xxkcv\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.292390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.292364    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5615fc2c-e1d3-41fb-a02e-2c88713a3900-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.292638 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:32.292619    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-serving-cert: secret "maas-api-e2e-inf-a-5121f4-serving-cert" not found
Sep 10 14:40:32.292706 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:32.292681    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls podName:5615fc2c-e1d3-41fb-a02e-2c88713a3900 nodeName:}" failed. No retries permitted until 2026-09-10 14:40:32.7926616 +0000 UTC m=+762.148474487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls") pod "maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" (UID: "5615fc2c-e1d3-41fb-a02e-2c88713a3900") : secret "maas-api-e2e-inf-a-5121f4-serving-cert" not found
Sep 10 14:40:32.295180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.295159    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-metrics-certs\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.298813 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.298789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxkcv\" (UniqueName: \"kubernetes.io/projected/5615fc2c-e1d3-41fb-a02e-2c88713a3900-kube-api-access-xxkcv\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.461187 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.461163    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"]
Sep 10 14:40:32.463145 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:32.463109    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2116c427_8cb4_4383_95ce_4386943e5546.slice/crio-3fe0507c22df9bc4ebaa0e14f2c38045e3e739bc2cec11aa7db07026fa29dc55 WatchSource:0}: Error finding container 3fe0507c22df9bc4ebaa0e14f2c38045e3e739bc2cec11aa7db07026fa29dc55: Status 404 returned error can't find the container with id 3fe0507c22df9bc4ebaa0e14f2c38045e3e739bc2cec11aa7db07026fa29dc55
Sep 10 14:40:32.694898 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.694873    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:32.796060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.796026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.798868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.798830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls\") pod \"maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:32.853640 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.853590    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:32.946300 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:32.946253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:33.058801 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.058701    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:33.210779 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.210731    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl" event={"ID":"2116c427-8cb4-4383-95ce-4386943e5546","Type":"ContainerStarted","Data":"9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb"}
Sep 10 14:40:33.211248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.210846    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:33.211248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.210881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl" event={"ID":"2116c427-8cb4-4383-95ce-4386943e5546","Type":"ContainerStarted","Data":"3fe0507c22df9bc4ebaa0e14f2c38045e3e739bc2cec11aa7db07026fa29dc55"}
Sep 10 14:40:33.212678 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.212650    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" event={"ID":"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a","Type":"ContainerStarted","Data":"aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130"}
Sep 10 14:40:33.212821 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.212702    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" event={"ID":"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a","Type":"ContainerStarted","Data":"9807f9723083f80099b84593037c208c3a5b51146370444130e4005ff9a6eea8"}
Sep 10 14:40:33.213055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.213033    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:33.227931 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.227883    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl" podStartSLOduration=2.227869033 podStartE2EDuration="2.227869033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:33.224409786 +0000 UTC m=+762.580222704" watchObservedRunningTime="2026-09-10 14:40:33.227869033 +0000 UTC m=+762.583681939"
Sep 10 14:40:33.246465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.246405    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" podStartSLOduration=1.814421584 podStartE2EDuration="2.246385492s" totalImagesPullingTime="431.963908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:33.244604109 +0000 UTC m=+762.600417036" watchObservedRunningTime="2026-09-10 14:40:33.246385492 +0000 UTC m=+762.602198402"
Sep 10 14:40:33.279111 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.279073    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:33.347105 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.347076    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"]
Sep 10 14:40:33.347959 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:33.347935    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5615fc2c_e1d3_41fb_a02e_2c88713a3900.slice/crio-e421243ab126ae14185c366f8f461b63d117316ad55a61e7c284429b2f251b72 WatchSource:0}: Error finding container e421243ab126ae14185c366f8f461b63d117316ad55a61e7c284429b2f251b72: Status 404 returned error can't find the container with id e421243ab126ae14185c366f8f461b63d117316ad55a61e7c284429b2f251b72
Sep 10 14:40:33.776532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.776491    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:33.925231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:33.925204    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:34.017736 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.017704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:34.223098 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.223058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" event={"ID":"5615fc2c-e1d3-41fb-a02e-2c88713a3900","Type":"ContainerStarted","Data":"8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9"}
Sep 10 14:40:34.223790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.223763    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:34.223864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.223809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" event={"ID":"5615fc2c-e1d3-41fb-a02e-2c88713a3900","Type":"ContainerStarted","Data":"e421243ab126ae14185c366f8f461b63d117316ad55a61e7c284429b2f251b72"}
Sep 10 14:40:34.244953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.244883    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" podStartSLOduration=1.916937653 podStartE2EDuration="2.244862457s" totalImagesPullingTime="327.924804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:34.244045433 +0000 UTC m=+763.599858361" watchObservedRunningTime="2026-09-10 14:40:34.244862457 +0000 UTC m=+763.600675361"
Sep 10 14:40:34.297169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.296366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:34.383104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.383012    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:34.521648 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:34.521620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:35.163405 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:35.163365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:36.236211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:36.236171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7830-d23a35" event={"ID":"e6af8578-ff63-4af8-b762-9d565681c8b4","Type":"ContainerStarted","Data":"d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42"}
Sep 10 14:40:36.249132 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:36.249080    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7830-d23a35" podStartSLOduration=1.683581099 podStartE2EDuration="9.249064328s" totalImagesPullingTime="7.565483229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:36.247873608 +0000 UTC m=+765.603686514" watchObservedRunningTime="2026-09-10 14:40:36.249064328 +0000 UTC m=+765.604877235"
Sep 10 14:40:36.843418 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:36.843377    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7830-d23a35"]
Sep 10 14:40:36.847059 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:36.847032    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7830-d23a35"]
Sep 10 14:40:37.242577 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:37.242546    2560 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7830-d23a35" container="test-curl-7830-d23a35"
Sep 10 14:40:37.318330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:37.318294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:37.416955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:37.416920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:38.218127 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.218101    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:40:38.249508 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.249476    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7830-d23a35" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" containerName="test-curl-7830-d23a35" containerID="cri-o://d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42" gracePeriod=2
Sep 10 14:40:38.436491 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.436453    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"]
Sep 10 14:40:38.437087 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.437069    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" containerName="test-curl-7830-d23a35"
Sep 10 14:40:38.437169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.437090    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" containerName="test-curl-7830-d23a35"
Sep 10 14:40:38.437246 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.437213    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" containerName="test-curl-7830-d23a35"
Sep 10 14:40:38.441202 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.441177    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.443651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.443628    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c2ce1a1c-serving-cert\""
Sep 10 14:40:38.453959 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.453923    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"]
Sep 10 14:40:38.475332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.475230    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:38.551187 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.551153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-maas-api-tls\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.551426 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.551219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-metrics-certs\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.551426 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.551311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-openshift-service-ca\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.551426 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.551340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5gc\" (UniqueName: \"kubernetes.io/projected/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-kube-api-access-fb5gc\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.572836 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.572794    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:38.652055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.652020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-maas-api-tls\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.652240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.652082    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-metrics-certs\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.652240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.652114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-openshift-service-ca\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.652240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.652135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5gc\" (UniqueName: \"kubernetes.io/projected/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-kube-api-access-fb5gc\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.652899 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.652871    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-openshift-service-ca\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.654583 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.654557    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-metrics-certs\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.654701 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.654683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-maas-api-tls\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.659443 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.659421    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5gc\" (UniqueName: \"kubernetes.io/projected/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-kube-api-access-fb5gc\") pod \"maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.673217 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.673188    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:38.753002 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.752909    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:38.951477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:38.951421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:39.029010 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.028985    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"]
Sep 10 14:40:39.030502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.030480    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:39.031290 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:39.031255    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a2b311_9f24_442d_b7d2_e84fdaa8fd7c.slice/crio-8875aafd304ff3ab71cbb791c4e3e0219c6bb7bb2f3657a897fab89b26451e6f WatchSource:0}: Error finding container 8875aafd304ff3ab71cbb791c4e3e0219c6bb7bb2f3657a897fab89b26451e6f: Status 404 returned error can't find the container with id 8875aafd304ff3ab71cbb791c4e3e0219c6bb7bb2f3657a897fab89b26451e6f
Sep 10 14:40:39.051752 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.051722    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"]
Sep 10 14:40:39.056673 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.056652    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.059140 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.059118    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c2ce1a1c\""
Sep 10 14:40:39.059304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.059264    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c2ce1a1c-dockercfg-6c9pd\""
Sep 10 14:40:39.073055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.073028    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"]
Sep 10 14:40:39.157064 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.157026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qk7b\" (UniqueName: \"kubernetes.io/projected/ce1ad954-e307-45b7-8156-a0bc6d040b7e-kube-api-access-2qk7b\") pod \"payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.157340 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.157076    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ad954-e307-45b7-8156-a0bc6d040b7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.256705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.256664    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz" event={"ID":"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c","Type":"ContainerStarted","Data":"8875aafd304ff3ab71cbb791c4e3e0219c6bb7bb2f3657a897fab89b26451e6f"}
Sep 10 14:40:39.258102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.258080    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qk7b\" (UniqueName: \"kubernetes.io/projected/ce1ad954-e307-45b7-8156-a0bc6d040b7e-kube-api-access-2qk7b\") pod \"payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.258168 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.258126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ad954-e307-45b7-8156-a0bc6d040b7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.258840 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.258817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ad954-e307-45b7-8156-a0bc6d040b7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.265399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.265379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qk7b\" (UniqueName: \"kubernetes.io/projected/ce1ad954-e307-45b7-8156-a0bc6d040b7e-kube-api-access-2qk7b\") pod \"payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.367073 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.366978    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:39.641017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:39.640990    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"]
Sep 10 14:40:39.641760 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:39.641733    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1ad954_e307_45b7_8156_a0bc6d040b7e.slice/crio-1dfa21f3f3e60068d87b6f53d0fc4a8774c4f181949a4a09d5c4867ed2ec1925 WatchSource:0}: Error finding container 1dfa21f3f3e60068d87b6f53d0fc4a8774c4f181949a4a09d5c4867ed2ec1925: Status 404 returned error can't find the container with id 1dfa21f3f3e60068d87b6f53d0fc4a8774c4f181949a4a09d5c4867ed2ec1925
Sep 10 14:40:40.235008 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.234977    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:40:40.275760 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.275341    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv" event={"ID":"ce1ad954-e307-45b7-8156-a0bc6d040b7e","Type":"ContainerStarted","Data":"bf4f0716a881465ae271f728cee2e8e382e1478816a07d6ee83a2c6257cc2498"}
Sep 10 14:40:40.275760 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.275428    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv" event={"ID":"ce1ad954-e307-45b7-8156-a0bc6d040b7e","Type":"ContainerStarted","Data":"1dfa21f3f3e60068d87b6f53d0fc4a8774c4f181949a4a09d5c4867ed2ec1925"}
Sep 10 14:40:40.278337 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.278306    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz" event={"ID":"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c","Type":"ContainerStarted","Data":"7c3feab520a520417a3fc15943761cede9c980dfd1dd06e968c5ca41060656c6"}
Sep 10 14:40:40.278465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.278454    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:40.293094 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.293040    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv" podStartSLOduration=1.293019919 podStartE2EDuration="1.293019919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:40.290023766 +0000 UTC m=+769.645836700" watchObservedRunningTime="2026-09-10 14:40:40.293019919 +0000 UTC m=+769.648832828"
Sep 10 14:40:40.307866 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.307804    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz" podStartSLOduration=1.8239507069999998 podStartE2EDuration="2.307782522s" totalImagesPullingTime="483.831815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:40.304894987 +0000 UTC m=+769.660707895" watchObservedRunningTime="2026-09-10 14:40:40.307782522 +0000 UTC m=+769.663595431"
Sep 10 14:40:40.444134 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.444107    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:40.571644 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.571560    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cbwd\" (UniqueName: \"kubernetes.io/projected/e6af8578-ff63-4af8-b762-9d565681c8b4-kube-api-access-8cbwd\") pod \"e6af8578-ff63-4af8-b762-9d565681c8b4\" (UID: \"e6af8578-ff63-4af8-b762-9d565681c8b4\") "
Sep 10 14:40:40.573954 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.573925    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6af8578-ff63-4af8-b762-9d565681c8b4-kube-api-access-8cbwd" (OuterVolumeSpecName: "kube-api-access-8cbwd") pod "e6af8578-ff63-4af8-b762-9d565681c8b4" (UID: "e6af8578-ff63-4af8-b762-9d565681c8b4"). InnerVolumeSpecName "kube-api-access-8cbwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:40:40.672524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:40.672491    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cbwd\" (UniqueName: \"kubernetes.io/projected/e6af8578-ff63-4af8-b762-9d565681c8b4-kube-api-access-8cbwd\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:40:41.241916 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.241886    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:40:41.285416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.285381    2560 generic.go:356] "Generic (PLEG): container finished" podID="e6af8578-ff63-4af8-b762-9d565681c8b4" containerID="d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42" exitCode=137
Sep 10 14:40:41.285828 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.285434    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:41.285828 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.285495    2560 scope.go:117] "RemoveContainer" containerID="d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42"
Sep 10 14:40:41.286240 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.286217    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:41.299825 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.299762    2560 scope.go:117] "RemoveContainer" containerID="d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42"
Sep 10 14:40:41.300190 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:41.300168    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42\": container with ID starting with d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42 not found: ID does not exist" containerID="d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42"
Sep 10 14:40:41.300304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.300197    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42"} err="failed to get container status \"d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42\": rpc error: code = NotFound desc = could not find container \"d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42\": container with ID starting with d0938110237248f3a5da77a47f985563fe6944280251a82d885476e52356ea42 not found: ID does not exist"
Sep 10 14:40:41.302065 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:41.302033    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7830-d23a35\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:41.317147 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:40:41.317117    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7830-d23a35\" is forbidden: User \"system:node:ip-10-0-132-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-253.ec2.internal' and this object" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" pod="opendatahub/test-curl-7830-d23a35"
Sep 10 14:40:41.320478 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.320451    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6af8578-ff63-4af8-b762-9d565681c8b4" path="/var/lib/kubelet/pods/e6af8578-ff63-4af8-b762-9d565681c8b4/volumes"
Sep 10 14:40:41.450530 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.450499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:41.752052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.752018    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:41.956762 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:41.956728    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:43.085532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:43.085498    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:40:43.905743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:43.905709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:44.147262 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:44.147228    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:44.225953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:44.225923    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:40:44.226132 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:44.226001    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:40:44.238530 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:44.238490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:44.342510 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:44.342478    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:45.285877 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:45.285848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:45.372780 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:45.372747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:46.291028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:46.291000    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:40:46.579054 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:46.578979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:46.680370 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:46.680341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:46.972765 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:46.972707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:47.074618 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.074589    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:47.173385 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.173355    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:47.273203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.273109    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:47.277932 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.277903    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"]
Sep 10 14:40:47.282051 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.282036    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.284822 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.284799    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-22fb28-dockercfg-8hlnx\""
Sep 10 14:40:47.285253 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.285233    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-22fb28\""
Sep 10 14:40:47.291423 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.291398    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"]
Sep 10 14:40:47.335577 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.335546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m66r\" (UniqueName: \"kubernetes.io/projected/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-kube-api-access-7m66r\") pod \"payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.335730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.335667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.436864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.436823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.437038 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.436913    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m66r\" (UniqueName: \"kubernetes.io/projected/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-kube-api-access-7m66r\") pod \"payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.437519 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.437499    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.444861 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.444835    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m66r\" (UniqueName: \"kubernetes.io/projected/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-kube-api-access-7m66r\") pod \"payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.594103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.594018    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:47.866609 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:47.866579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"]
Sep 10 14:40:47.868265 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:40:47.868239    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0ee985_2f7b_4545_ae85_c7fd6091cf91.slice/crio-647db71f8ddaf534861fbc44b6fd2f5f4043ac137352eaaea5920918ba235c9b WatchSource:0}: Error finding container 647db71f8ddaf534861fbc44b6fd2f5f4043ac137352eaaea5920918ba235c9b: Status 404 returned error can't find the container with id 647db71f8ddaf534861fbc44b6fd2f5f4043ac137352eaaea5920918ba235c9b
Sep 10 14:40:48.334397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:48.334346    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x" event={"ID":"ec0ee985-2f7b-4545-ae85-c7fd6091cf91","Type":"ContainerStarted","Data":"c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61"}
Sep 10 14:40:48.334766 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:48.334442    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x" event={"ID":"ec0ee985-2f7b-4545-ae85-c7fd6091cf91","Type":"ContainerStarted","Data":"647db71f8ddaf534861fbc44b6fd2f5f4043ac137352eaaea5920918ba235c9b"}
Sep 10 14:40:48.334766 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:48.334502    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:48.351410 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:48.351363    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x" podStartSLOduration=1.351348555 podStartE2EDuration="1.351348555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:48.348638888 +0000 UTC m=+777.704451815" watchObservedRunningTime="2026-09-10 14:40:48.351348555 +0000 UTC m=+777.707161462"
Sep 10 14:40:50.663505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:50.663469    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.170669 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:51.170632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:52.167761 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:52.167720    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:52.259041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:52.259005    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:52.295856 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:52.295817    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:40:52.454995 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:52.454961    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:52.552332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:52.552302    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:53.175261 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:53.175219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:53.846393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:53.846361    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:53.947241 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:53.947195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:54.333081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:54.333047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:54.430679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:54.430642    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:54.530999 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:54.530973    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:54.689558 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:54.689529    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:55.553671 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:55.553640    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.405018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:58.404983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.465311 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:58.465256    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.540443 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:58.540407    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.640165 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:58.640128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.757013 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:58.756986    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.999955 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:58.999909    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:59.342452 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:59.342422    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:40:59.344844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:59.344801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:59.444859 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:59.444826    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:59.665982 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:59.665904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:59.765781 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:40:59.765749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:03.175040 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:03.175004    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:03.269885 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:03.269851    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:03.375580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:03.375540    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.339996 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:04.339941    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.406810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:04.406780    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.507334 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:04.507302    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.818724 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:04.818697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.886999 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:04.886966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:05.249910 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:05.249878    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:05.349523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:05.349493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:06.182511 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:06.182469    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:41:08.519116 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:08.519081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:10.239125 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:10.239078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:18.654670 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:18.654598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:19.213730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.213699    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"]
Sep 10 14:41:19.214034 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.214003    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz" podUID="30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" containerName="maas-api" containerID="cri-o://7c3feab520a520417a3fc15943761cede9c980dfd1dd06e968c5ca41060656c6" gracePeriod=30
Sep 10 14:41:19.244936 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.244909    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:19.279821 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.279680    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"]
Sep 10 14:41:19.280447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.280208    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv" podUID="ce1ad954-e307-45b7-8156-a0bc6d040b7e" containerName="payload-pre-processing" containerID="cri-o://bf4f0716a881465ae271f728cee2e8e382e1478816a07d6ee83a2c6257cc2498" gracePeriod=30
Sep 10 14:41:19.344402 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.344372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:19.518706 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.518680    2560 generic.go:356] "Generic (PLEG): container finished" podID="ce1ad954-e307-45b7-8156-a0bc6d040b7e" containerID="bf4f0716a881465ae271f728cee2e8e382e1478816a07d6ee83a2c6257cc2498" exitCode=0
Sep 10 14:41:19.518870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.518748    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv" event={"ID":"ce1ad954-e307-45b7-8156-a0bc6d040b7e","Type":"ContainerDied","Data":"bf4f0716a881465ae271f728cee2e8e382e1478816a07d6ee83a2c6257cc2498"}
Sep 10 14:41:19.520703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.520684    2560 generic.go:356] "Generic (PLEG): container finished" podID="30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" containerID="7c3feab520a520417a3fc15943761cede9c980dfd1dd06e968c5ca41060656c6" exitCode=0
Sep 10 14:41:19.520810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.520712    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz" event={"ID":"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c","Type":"ContainerDied","Data":"7c3feab520a520417a3fc15943761cede9c980dfd1dd06e968c5ca41060656c6"}
Sep 10 14:41:19.537678 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.537550    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:19.562047 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.562025    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:41:19.564965 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.564944    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:41:19.601737 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.601712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:19.626081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626045    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-metrics-certs\") pod \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") "
Sep 10 14:41:19.626247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626106    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb5gc\" (UniqueName: \"kubernetes.io/projected/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-kube-api-access-fb5gc\") pod \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") "
Sep 10 14:41:19.626247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626146    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qk7b\" (UniqueName: \"kubernetes.io/projected/ce1ad954-e307-45b7-8156-a0bc6d040b7e-kube-api-access-2qk7b\") pod \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") "
Sep 10 14:41:19.626247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626206    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ad954-e307-45b7-8156-a0bc6d040b7e-plugins-config-volume\") pod \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\" (UID: \"ce1ad954-e307-45b7-8156-a0bc6d040b7e\") "
Sep 10 14:41:19.626247 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626243    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-maas-api-tls\") pod \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") "
Sep 10 14:41:19.626500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626307    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-openshift-service-ca\") pod \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\" (UID: \"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c\") "
Sep 10 14:41:19.626824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626792    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce1ad954-e307-45b7-8156-a0bc6d040b7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce1ad954-e307-45b7-8156-a0bc6d040b7e" (UID: "ce1ad954-e307-45b7-8156-a0bc6d040b7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:19.626975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.626951    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" (UID: "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:19.628484 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.628449    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1ad954-e307-45b7-8156-a0bc6d040b7e-kube-api-access-2qk7b" (OuterVolumeSpecName: "kube-api-access-2qk7b") pod "ce1ad954-e307-45b7-8156-a0bc6d040b7e" (UID: "ce1ad954-e307-45b7-8156-a0bc6d040b7e"). InnerVolumeSpecName "kube-api-access-2qk7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:19.628576 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.628521    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" (UID: "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:19.628617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.628572    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" (UID: "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:19.628764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.628740    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-kube-api-access-fb5gc" (OuterVolumeSpecName: "kube-api-access-fb5gc") pod "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" (UID: "30a2b311-9f24-442d-b7d2-e84fdaa8fd7c"). InnerVolumeSpecName "kube-api-access-fb5gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:19.668815 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.668785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:19.727518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.727487    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ad954-e307-45b7-8156-a0bc6d040b7e-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:19.727518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.727513    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:19.727518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.727523    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:19.727735 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.727531    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:19.727735 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.727540    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb5gc\" (UniqueName: \"kubernetes.io/projected/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c-kube-api-access-fb5gc\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:19.727735 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:19.727547    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qk7b\" (UniqueName: \"kubernetes.io/projected/ce1ad954-e307-45b7-8156-a0bc6d040b7e-kube-api-access-2qk7b\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:20.193320 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.193264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:20.300217 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.300157    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:20.393177 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.393145    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:20.527304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.527214    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"
Sep 10 14:41:20.527304 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.527239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz" event={"ID":"30a2b311-9f24-442d-b7d2-e84fdaa8fd7c","Type":"ContainerDied","Data":"8875aafd304ff3ab71cbb791c4e3e0219c6bb7bb2f3657a897fab89b26451e6f"}
Sep 10 14:41:20.527527 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.527314    2560 scope.go:117] "RemoveContainer" containerID="7c3feab520a520417a3fc15943761cede9c980dfd1dd06e968c5ca41060656c6"
Sep 10 14:41:20.528844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.528809    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"
Sep 10 14:41:20.528844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.528833    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv" event={"ID":"ce1ad954-e307-45b7-8156-a0bc6d040b7e","Type":"ContainerDied","Data":"1dfa21f3f3e60068d87b6f53d0fc4a8774c4f181949a4a09d5c4867ed2ec1925"}
Sep 10 14:41:20.537288 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.537257    2560 scope.go:117] "RemoveContainer" containerID="bf4f0716a881465ae271f728cee2e8e382e1478816a07d6ee83a2c6257cc2498"
Sep 10 14:41:20.560158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.560137    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"]
Sep 10 14:41:20.563757 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.563730    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2ce1a1c-6464749869-cgnrz"]
Sep 10 14:41:20.574932 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.574905    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"]
Sep 10 14:41:20.581701 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.581683    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2ce1a1c-5c9f8bb766-rwhzv"]
Sep 10 14:41:20.699482 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.699447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:20.799029 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:20.798941    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:21.319170 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:21.319134    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" path="/var/lib/kubelet/pods/30a2b311-9f24-442d-b7d2-e84fdaa8fd7c/volumes"
Sep 10 14:41:21.319618 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:21.319596    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce1ad954-e307-45b7-8156-a0bc6d040b7e" path="/var/lib/kubelet/pods/ce1ad954-e307-45b7-8156-a0bc6d040b7e/volumes"
Sep 10 14:41:22.949364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:22.949319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.009865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.009832    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.052835 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.052796    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"]
Sep 10 14:41:23.053603 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.053581    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce1ad954-e307-45b7-8156-a0bc6d040b7e" containerName="payload-pre-processing"
Sep 10 14:41:23.053668 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.053607    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce1ad954-e307-45b7-8156-a0bc6d040b7e" containerName="payload-pre-processing"
Sep 10 14:41:23.053668 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.053627    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" containerName="maas-api"
Sep 10 14:41:23.053668 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.053636    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" containerName="maas-api"
Sep 10 14:41:23.053777 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.053764    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce1ad954-e307-45b7-8156-a0bc6d040b7e" containerName="payload-pre-processing"
Sep 10 14:41:23.053817 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.053781    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a2b311-9f24-442d-b7d2-e84fdaa8fd7c" containerName="maas-api"
Sep 10 14:41:23.059025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.058991    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.062148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.062121    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-42b72890-serving-cert\""
Sep 10 14:41:23.065095 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.065068    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"]
Sep 10 14:41:23.098509 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.098477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.156436 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.156400    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/495ac407-c41c-4d62-a9f6-cfb2204728c9-openshift-service-ca\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.156596 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.156454    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-metrics-certs\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.156596 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.156573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-maas-api-tls\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.156704 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.156631    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98mnr\" (UniqueName: \"kubernetes.io/projected/495ac407-c41c-4d62-a9f6-cfb2204728c9-kube-api-access-98mnr\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.198707 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.198674    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.257845 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.257758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/495ac407-c41c-4d62-a9f6-cfb2204728c9-openshift-service-ca\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.257845 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.257804    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-metrics-certs\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.257845 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.257838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-maas-api-tls\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.258100 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.257863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98mnr\" (UniqueName: \"kubernetes.io/projected/495ac407-c41c-4d62-a9f6-cfb2204728c9-kube-api-access-98mnr\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.258563 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.258534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/495ac407-c41c-4d62-a9f6-cfb2204728c9-openshift-service-ca\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.260290 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.260259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-metrics-certs\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.260372 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.260327    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-maas-api-tls\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.265649 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.265627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98mnr\" (UniqueName: \"kubernetes.io/projected/495ac407-c41c-4d62-a9f6-cfb2204728c9-kube-api-access-98mnr\") pod \"maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.299289 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.299252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.374830 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.374795    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:23.593481 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.593403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.661688 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.661658    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"]
Sep 10 14:41:23.662775 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.662743    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:23.663003 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:41:23.662977    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495ac407_c41c_4d62_a9f6_cfb2204728c9.slice/crio-61ea41b8f34563c2375ea99c3d682e9683197d8c2e76098ac7fc71756b3a2409 WatchSource:0}: Error finding container 61ea41b8f34563c2375ea99c3d682e9683197d8c2e76098ac7fc71756b3a2409: Status 404 returned error can't find the container with id 61ea41b8f34563c2375ea99c3d682e9683197d8c2e76098ac7fc71756b3a2409
Sep 10 14:41:23.685455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.685432    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"]
Sep 10 14:41:23.690412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.690387    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.692392 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.692375    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-42b72890\""
Sep 10 14:41:23.696755 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.696733    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"]
Sep 10 14:41:23.729531 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.729505    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"]
Sep 10 14:41:23.733121 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.733102    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:23.735408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.735379    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-42b72890-dockercfg-r8wpc\""
Sep 10 14:41:23.745012 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.744989    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"]
Sep 10 14:41:23.762469 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.762447    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.762666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.762498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0699e5dc-5558-464d-b27c-3033057d48d2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:23.762666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.762562    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrh7z\" (UniqueName: \"kubernetes.io/projected/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-kube-api-access-jrh7z\") pod \"payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.762666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.762601    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5xn\" (UniqueName: \"kubernetes.io/projected/0699e5dc-5558-464d-b27c-3033057d48d2-kube-api-access-dd5xn\") pod \"payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:23.863710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.863624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0699e5dc-5558-464d-b27c-3033057d48d2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:23.863710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.863667    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrh7z\" (UniqueName: \"kubernetes.io/projected/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-kube-api-access-jrh7z\") pod \"payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.863710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.863697    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5xn\" (UniqueName: \"kubernetes.io/projected/0699e5dc-5558-464d-b27c-3033057d48d2-kube-api-access-dd5xn\") pod \"payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:23.863999 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.863752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.864477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.864450    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0699e5dc-5558-464d-b27c-3033057d48d2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:23.864607 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.864587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.875720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.875692    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrh7z\" (UniqueName: \"kubernetes.io/projected/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-kube-api-access-jrh7z\") pod \"payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:23.875824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:23.875738    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5xn\" (UniqueName: \"kubernetes.io/projected/0699e5dc-5558-464d-b27c-3033057d48d2-kube-api-access-dd5xn\") pod \"payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:24.002296 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.002253    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:24.043157 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.043122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:24.282623 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.279006    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"]
Sep 10 14:41:24.333237 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.333215    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"]
Sep 10 14:41:24.334598 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:41:24.334574    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0699e5dc_5558_464d_b27c_3033057d48d2.slice/crio-ceca05728fbca6e2123855ff749419e8f4ae9e54a0a5fe6f52398c451aff6c80 WatchSource:0}: Error finding container ceca05728fbca6e2123855ff749419e8f4ae9e54a0a5fe6f52398c451aff6c80: Status 404 returned error can't find the container with id ceca05728fbca6e2123855ff749419e8f4ae9e54a0a5fe6f52398c451aff6c80
Sep 10 14:41:24.557021 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.556987    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz" event={"ID":"495ac407-c41c-4d62-a9f6-cfb2204728c9","Type":"ContainerStarted","Data":"a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112"}
Sep 10 14:41:24.557228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.557037    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz" event={"ID":"495ac407-c41c-4d62-a9f6-cfb2204728c9","Type":"ContainerStarted","Data":"61ea41b8f34563c2375ea99c3d682e9683197d8c2e76098ac7fc71756b3a2409"}
Sep 10 14:41:24.557228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.557188    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:24.558952 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.558927    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz" event={"ID":"0699e5dc-5558-464d-b27c-3033057d48d2","Type":"ContainerStarted","Data":"19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10"}
Sep 10 14:41:24.559072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.558973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz" event={"ID":"0699e5dc-5558-464d-b27c-3033057d48d2","Type":"ContainerStarted","Data":"ceca05728fbca6e2123855ff749419e8f4ae9e54a0a5fe6f52398c451aff6c80"}
Sep 10 14:41:24.559072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.559006    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:24.560648 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.560624    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr" event={"ID":"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe","Type":"ContainerStarted","Data":"b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f"}
Sep 10 14:41:24.560743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.560667    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr" event={"ID":"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe","Type":"ContainerStarted","Data":"2e4f75e23109408261f18cecff500aa1f67b3464053cd856d736ed1f6707b5df"}
Sep 10 14:41:24.560743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.560708    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:24.574331 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.574265    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz" podStartSLOduration=1.053925641 podStartE2EDuration="1.574248865s" totalImagesPullingTime="520.323224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:24.571940004 +0000 UTC m=+813.927752910" watchObservedRunningTime="2026-09-10 14:41:24.574248865 +0000 UTC m=+813.930061773"
Sep 10 14:41:24.588001 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.587952    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz" podStartSLOduration=1.5879352089999998 podStartE2EDuration="1.587935209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:24.587027351 +0000 UTC m=+813.942840257" watchObservedRunningTime="2026-09-10 14:41:24.587935209 +0000 UTC m=+813.943748114"
Sep 10 14:41:24.604130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:24.604087    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr" podStartSLOduration=1.604073826 podStartE2EDuration="1.604073826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:24.602601221 +0000 UTC m=+813.958414149" watchObservedRunningTime="2026-09-10 14:41:24.604073826 +0000 UTC m=+813.959886732"
Sep 10 14:41:25.236652 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:25.236617    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:26.077455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:26.077419    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:41:28.148925 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:28.148898    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:30.572214 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:30.572176    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:32.420193 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:32.420154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:35.568970 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:35.568937    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:35.569387 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:35.569020    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:42.895212 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:42.895165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:43.980739 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:43.980703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:46.381108 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:46.381074    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:41:50.126246 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.126213    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"]
Sep 10 14:41:50.129890 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.129866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.132128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.132109    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9b1e67\"/\"openshift-service-ca.crt\""
Sep 10 14:41:50.132738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.132717    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9b1e67\"/\"simulator-w0-9b1e67-kserve-self-signed-certs\""
Sep 10 14:41:50.132839 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.132801    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9b1e67\"/\"default-dockercfg-cx7lb\""
Sep 10 14:41:50.132839 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.132817    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9b1e67\"/\"kube-root-ca.crt\""
Sep 10 14:41:50.142228 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.142203    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"]
Sep 10 14:41:50.154263 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.154242    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:50.317566 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.317526    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tls-certs\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.317764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.317579    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-home\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.317764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.317650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-model-cache\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.317764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.317683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-dshm\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.317764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.317713    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tmp-dir\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.317764 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.317752    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4gkr\" (UniqueName: \"kubernetes.io/projected/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-kube-api-access-g4gkr\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.418916 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.418822    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4gkr\" (UniqueName: \"kubernetes.io/projected/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-kube-api-access-g4gkr\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.418916 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.418883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tls-certs\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.418997    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-home\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.419059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-model-cache\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.419092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-dshm\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.419133    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tmp-dir\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419491 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.419468    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-home\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419584 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.419561    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-model-cache\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.419642 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.419597    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tmp-dir\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.421342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.421312    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-dshm\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.421651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.421631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tls-certs\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.426132 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.426108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4gkr\" (UniqueName: \"kubernetes.io/projected/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-kube-api-access-g4gkr\") pod \"simulator-w0-9b1e67-kserve-57d9b948c-z5ntc\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") " pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.440156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.440132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:41:50.719618 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:50.719580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"]
Sep 10 14:41:50.720787 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:41:50.720762    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod136ceb19_3aaf_4d94_beba_1dc83e8fcbab.slice/crio-14e13b58a9353bc9a3bacbcfb9e508f9636bcce2049b9f0759be4fd47b02c24f WatchSource:0}: Error finding container 14e13b58a9353bc9a3bacbcfb9e508f9636bcce2049b9f0759be4fd47b02c24f: Status 404 returned error can't find the container with id 14e13b58a9353bc9a3bacbcfb9e508f9636bcce2049b9f0759be4fd47b02c24f
Sep 10 14:41:51.720296 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:51.720249    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc" event={"ID":"136ceb19-3aaf-4d94-beba-1dc83e8fcbab","Type":"ContainerStarted","Data":"316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc"}
Sep 10 14:41:51.720706 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:51.720308    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc" event={"ID":"136ceb19-3aaf-4d94-beba-1dc83e8fcbab","Type":"ContainerStarted","Data":"14e13b58a9353bc9a3bacbcfb9e508f9636bcce2049b9f0759be4fd47b02c24f"}
Sep 10 14:41:51.737666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:51.737625    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc" podStartSLOduration=1.737610806 podStartE2EDuration="1.737610806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:51.73455142 +0000 UTC m=+841.090364326" watchObservedRunningTime="2026-09-10 14:41:51.737610806 +0000 UTC m=+841.093423712"
Sep 10 14:41:53.825767 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.825723    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"]
Sep 10 14:41:53.826226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.825979    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz" podUID="495ac407-c41c-4d62-a9f6-cfb2204728c9" containerName="maas-api" containerID="cri-o://a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112" gracePeriod=30
Sep 10 14:41:53.857048 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.857016    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:53.889901 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.889866    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"]
Sep 10 14:41:53.890319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.890267    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz" podUID="0699e5dc-5558-464d-b27c-3033057d48d2" containerName="payload-processing" containerID="cri-o://19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10" gracePeriod=30
Sep 10 14:41:53.897689 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.897661    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"]
Sep 10 14:41:53.897891 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.897834    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr" podUID="3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" containerName="payload-pre-processing" containerID="cri-o://b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f" gracePeriod=30
Sep 10 14:41:53.931731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.931702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:53.999420 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:53.999392    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:54.095337 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.095251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:54.161128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.161088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:54.164103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.164079    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:54.191140 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.190826    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:54.246984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.246953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:54.248142 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.248124    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:54.259319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259260    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-maas-api-tls\") pod \"495ac407-c41c-4d62-a9f6-cfb2204728c9\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") "
Sep 10 14:41:54.259526 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259507    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-plugins-config-volume\") pod \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") "
Sep 10 14:41:54.259601 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259555    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrh7z\" (UniqueName: \"kubernetes.io/projected/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-kube-api-access-jrh7z\") pod \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\" (UID: \"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe\") "
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259792    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0699e5dc-5558-464d-b27c-3033057d48d2-plugins-config-volume\") pod \"0699e5dc-5558-464d-b27c-3033057d48d2\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") "
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259849    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/495ac407-c41c-4d62-a9f6-cfb2204728c9-openshift-service-ca\") pod \"495ac407-c41c-4d62-a9f6-cfb2204728c9\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") "
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259874    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-metrics-certs\") pod \"495ac407-c41c-4d62-a9f6-cfb2204728c9\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") "
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259911    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd5xn\" (UniqueName: \"kubernetes.io/projected/0699e5dc-5558-464d-b27c-3033057d48d2-kube-api-access-dd5xn\") pod \"0699e5dc-5558-464d-b27c-3033057d48d2\" (UID: \"0699e5dc-5558-464d-b27c-3033057d48d2\") "
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.259953    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98mnr\" (UniqueName: \"kubernetes.io/projected/495ac407-c41c-4d62-a9f6-cfb2204728c9-kube-api-access-98mnr\") pod \"495ac407-c41c-4d62-a9f6-cfb2204728c9\" (UID: \"495ac407-c41c-4d62-a9f6-cfb2204728c9\") "
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.260164    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" (UID: "3f09c915-efb4-4b32-bf9f-fc5a6751d1fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.260256    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495ac407-c41c-4d62-a9f6-cfb2204728c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "495ac407-c41c-4d62-a9f6-cfb2204728c9" (UID: "495ac407-c41c-4d62-a9f6-cfb2204728c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:54.260520 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.260270    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.260990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.260486    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0699e5dc-5558-464d-b27c-3033057d48d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0699e5dc-5558-464d-b27c-3033057d48d2" (UID: "0699e5dc-5558-464d-b27c-3033057d48d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:54.262272 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.262240    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-kube-api-access-jrh7z" (OuterVolumeSpecName: "kube-api-access-jrh7z") pod "3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" (UID: "3f09c915-efb4-4b32-bf9f-fc5a6751d1fe"). InnerVolumeSpecName "kube-api-access-jrh7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:54.262615 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.262580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0699e5dc-5558-464d-b27c-3033057d48d2-kube-api-access-dd5xn" (OuterVolumeSpecName: "kube-api-access-dd5xn") pod "0699e5dc-5558-464d-b27c-3033057d48d2" (UID: "0699e5dc-5558-464d-b27c-3033057d48d2"). InnerVolumeSpecName "kube-api-access-dd5xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:54.262703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.262608    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "495ac407-c41c-4d62-a9f6-cfb2204728c9" (UID: "495ac407-c41c-4d62-a9f6-cfb2204728c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:54.262703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.262669    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/495ac407-c41c-4d62-a9f6-cfb2204728c9-kube-api-access-98mnr" (OuterVolumeSpecName: "kube-api-access-98mnr") pod "495ac407-c41c-4d62-a9f6-cfb2204728c9" (UID: "495ac407-c41c-4d62-a9f6-cfb2204728c9"). InnerVolumeSpecName "kube-api-access-98mnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:54.262897 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.262880    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "495ac407-c41c-4d62-a9f6-cfb2204728c9" (UID: "495ac407-c41c-4d62-a9f6-cfb2204728c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:54.361342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361214    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0699e5dc-5558-464d-b27c-3033057d48d2-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.361342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361253    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/495ac407-c41c-4d62-a9f6-cfb2204728c9-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.361342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361270    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.361342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361313    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd5xn\" (UniqueName: \"kubernetes.io/projected/0699e5dc-5558-464d-b27c-3033057d48d2-kube-api-access-dd5xn\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.361342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361328    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98mnr\" (UniqueName: \"kubernetes.io/projected/495ac407-c41c-4d62-a9f6-cfb2204728c9-kube-api-access-98mnr\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.361342 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361343    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/495ac407-c41c-4d62-a9f6-cfb2204728c9-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.361669 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.361357    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrh7z\" (UniqueName: \"kubernetes.io/projected/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe-kube-api-access-jrh7z\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:41:54.710682 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.710644    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:54.739060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.739028    2560 generic.go:356] "Generic (PLEG): container finished" podID="495ac407-c41c-4d62-a9f6-cfb2204728c9" containerID="a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112" exitCode=0
Sep 10 14:41:54.739232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.739089    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"
Sep 10 14:41:54.739232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.739111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz" event={"ID":"495ac407-c41c-4d62-a9f6-cfb2204728c9","Type":"ContainerDied","Data":"a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112"}
Sep 10 14:41:54.739232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.739164    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz" event={"ID":"495ac407-c41c-4d62-a9f6-cfb2204728c9","Type":"ContainerDied","Data":"61ea41b8f34563c2375ea99c3d682e9683197d8c2e76098ac7fc71756b3a2409"}
Sep 10 14:41:54.739232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.739196    2560 scope.go:117] "RemoveContainer" containerID="a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112"
Sep 10 14:41:54.740602 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.740574    2560 generic.go:356] "Generic (PLEG): container finished" podID="0699e5dc-5558-464d-b27c-3033057d48d2" containerID="19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10" exitCode=0
Sep 10 14:41:54.740703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.740624    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"
Sep 10 14:41:54.740703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.740636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz" event={"ID":"0699e5dc-5558-464d-b27c-3033057d48d2","Type":"ContainerDied","Data":"19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10"}
Sep 10 14:41:54.740703 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.740680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz" event={"ID":"0699e5dc-5558-464d-b27c-3033057d48d2","Type":"ContainerDied","Data":"ceca05728fbca6e2123855ff749419e8f4ae9e54a0a5fe6f52398c451aff6c80"}
Sep 10 14:41:54.742389 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.742364    2560 generic.go:356] "Generic (PLEG): container finished" podID="3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" containerID="b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f" exitCode=0
Sep 10 14:41:54.742518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.742446    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr" event={"ID":"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe","Type":"ContainerDied","Data":"b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f"}
Sep 10 14:41:54.742518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.742465    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"
Sep 10 14:41:54.742518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.742497    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr" event={"ID":"3f09c915-efb4-4b32-bf9f-fc5a6751d1fe","Type":"ContainerDied","Data":"2e4f75e23109408261f18cecff500aa1f67b3464053cd856d736ed1f6707b5df"}
Sep 10 14:41:54.750168 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.750153    2560 scope.go:117] "RemoveContainer" containerID="a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112"
Sep 10 14:41:54.750509 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:41:54.750488    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112\": container with ID starting with a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112 not found: ID does not exist" containerID="a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112"
Sep 10 14:41:54.750589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.750514    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112"} err="failed to get container status \"a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112\": rpc error: code = NotFound desc = could not find container \"a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112\": container with ID starting with a60d6904e8e47f5fe6bf497711b9d42cc105b93c64adefdf351cf89c9c4d7112 not found: ID does not exist"
Sep 10 14:41:54.750589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.750530    2560 scope.go:117] "RemoveContainer" containerID="19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10"
Sep 10 14:41:54.759332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.759315    2560 scope.go:117] "RemoveContainer" containerID="19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10"
Sep 10 14:41:54.759595 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:41:54.759579    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10\": container with ID starting with 19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10 not found: ID does not exist" containerID="19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10"
Sep 10 14:41:54.759643 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.759602    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10"} err="failed to get container status \"19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10\": rpc error: code = NotFound desc = could not find container \"19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10\": container with ID starting with 19028cb7fa021ec4695f5ccaae0e1c21911081d8928a75da90aa7744a9eaae10 not found: ID does not exist"
Sep 10 14:41:54.759643 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.759618    2560 scope.go:117] "RemoveContainer" containerID="b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f"
Sep 10 14:41:54.767697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.767680    2560 scope.go:117] "RemoveContainer" containerID="b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f"
Sep 10 14:41:54.767944 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:41:54.767928    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f\": container with ID starting with b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f not found: ID does not exist" containerID="b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f"
Sep 10 14:41:54.768009 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.767952    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f"} err="failed to get container status \"b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f\": rpc error: code = NotFound desc = could not find container \"b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f\": container with ID starting with b9d3e906a33379eb2a2b6544ac140b22bd67252d9a929c0237b36d9dd66c6d7f not found: ID does not exist"
Sep 10 14:41:54.777122 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.777098    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"]
Sep 10 14:41:54.779957 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.779939    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42b72890-779d5c7d5f-czwrz"]
Sep 10 14:41:54.789827 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.789808    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"]
Sep 10 14:41:54.793506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.793488    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b72890-5b4c896cb6-p7kpr"]
Sep 10 14:41:54.803158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.803137    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"]
Sep 10 14:41:54.806986 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:54.806967    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b72890-7b679b9f74-xnvpz"]
Sep 10 14:41:55.044107 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:55.044007    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:55.319460 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:55.319365    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0699e5dc-5558-464d-b27c-3033057d48d2" path="/var/lib/kubelet/pods/0699e5dc-5558-464d-b27c-3033057d48d2/volumes"
Sep 10 14:41:55.319832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:55.319810    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" path="/var/lib/kubelet/pods/3f09c915-efb4-4b32-bf9f-fc5a6751d1fe/volumes"
Sep 10 14:41:55.320235 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:55.320215    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="495ac407-c41c-4d62-a9f6-cfb2204728c9" path="/var/lib/kubelet/pods/495ac407-c41c-4d62-a9f6-cfb2204728c9/volumes"
Sep 10 14:41:55.447320 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:55.447263    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:57.166727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:57.166697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:57.239732 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:57.239700    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:58.263974 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.263641    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"]
Sep 10 14:41:58.264445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264312    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" containerName="payload-pre-processing"
Sep 10 14:41:58.264445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264328    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" containerName="payload-pre-processing"
Sep 10 14:41:58.264445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264364    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0699e5dc-5558-464d-b27c-3033057d48d2" containerName="payload-processing"
Sep 10 14:41:58.264445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264373    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0699e5dc-5558-464d-b27c-3033057d48d2" containerName="payload-processing"
Sep 10 14:41:58.264445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264389    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="495ac407-c41c-4d62-a9f6-cfb2204728c9" containerName="maas-api"
Sep 10 14:41:58.264445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264397    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="495ac407-c41c-4d62-a9f6-cfb2204728c9" containerName="maas-api"
Sep 10 14:41:58.264769 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264484    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0699e5dc-5558-464d-b27c-3033057d48d2" containerName="payload-processing"
Sep 10 14:41:58.264769 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264498    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="495ac407-c41c-4d62-a9f6-cfb2204728c9" containerName="maas-api"
Sep 10 14:41:58.264769 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.264510    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f09c915-efb4-4b32-bf9f-fc5a6751d1fe" containerName="payload-pre-processing"
Sep 10 14:41:58.269812 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.269790    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.272996 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.272959    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7b4382a5-serving-cert\""
Sep 10 14:41:58.280298 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.280254    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"]
Sep 10 14:41:58.294717 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.294680    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-metrics-certs\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.294874 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.294745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/978eee43-ae26-4500-8236-f39f2c3e123e-openshift-service-ca\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.294874 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.294814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swx7j\" (UniqueName: \"kubernetes.io/projected/978eee43-ae26-4500-8236-f39f2c3e123e-kube-api-access-swx7j\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.294981 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.294873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-maas-api-tls\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.300422 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.300385    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:58.370360 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.370324    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:58.395909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.395881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swx7j\" (UniqueName: \"kubernetes.io/projected/978eee43-ae26-4500-8236-f39f2c3e123e-kube-api-access-swx7j\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.396083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.395940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-maas-api-tls\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.396083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.396017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-metrics-certs\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.396083 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.396064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/978eee43-ae26-4500-8236-f39f2c3e123e-openshift-service-ca\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.396773 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.396748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/978eee43-ae26-4500-8236-f39f2c3e123e-openshift-service-ca\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.398475 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.398451    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-metrics-certs\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.398601 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.398563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-maas-api-tls\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.404718 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.404695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swx7j\" (UniqueName: \"kubernetes.io/projected/978eee43-ae26-4500-8236-f39f2c3e123e-kube-api-access-swx7j\") pod \"maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.585632 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.585562    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:58.769163 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.769132    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:58.859383 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.859352    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"]
Sep 10 14:41:58.860765 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:41:58.860733    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978eee43_ae26_4500_8236_f39f2c3e123e.slice/crio-2e715ae528db43d17ea7cecec59961fb243e3067ab6db5bf580437c99a6b8e6b WatchSource:0}: Error finding container 2e715ae528db43d17ea7cecec59961fb243e3067ab6db5bf580437c99a6b8e6b: Status 404 returned error can't find the container with id 2e715ae528db43d17ea7cecec59961fb243e3067ab6db5bf580437c99a6b8e6b
Sep 10 14:41:58.860966 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.860881    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:58.881683 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.881652    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"]
Sep 10 14:41:58.886378 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.886363    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:58.888787 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.888763    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7b4382a5\""
Sep 10 14:41:58.888880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.888867    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7b4382a5-dockercfg-5qz82\""
Sep 10 14:41:58.895730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.895708    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"]
Sep 10 14:41:58.900594 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.900568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39107e65-6e50-44ff-9943-ad7f8808eb53-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:58.900695 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:58.900620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsxh9\" (UniqueName: \"kubernetes.io/projected/39107e65-6e50-44ff-9943-ad7f8808eb53-kube-api-access-rsxh9\") pod \"payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.001750 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.001714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39107e65-6e50-44ff-9943-ad7f8808eb53-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.001750 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.001756    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsxh9\" (UniqueName: \"kubernetes.io/projected/39107e65-6e50-44ff-9943-ad7f8808eb53-kube-api-access-rsxh9\") pod \"payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.002462 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.002441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39107e65-6e50-44ff-9943-ad7f8808eb53-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.009618 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.009595    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsxh9\" (UniqueName: \"kubernetes.io/projected/39107e65-6e50-44ff-9943-ad7f8808eb53-kube-api-access-rsxh9\") pod \"payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.198473 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.198438    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.477564 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.477538    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"]
Sep 10 14:41:59.479342 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:41:59.479315    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39107e65_6e50_44ff_9943_ad7f8808eb53.slice/crio-52f074d03a59a941a2460649c5fb1fad64ea107b58732cb32a6f9fde771afcee WatchSource:0}: Error finding container 52f074d03a59a941a2460649c5fb1fad64ea107b58732cb32a6f9fde771afcee: Status 404 returned error can't find the container with id 52f074d03a59a941a2460649c5fb1fad64ea107b58732cb32a6f9fde771afcee
Sep 10 14:41:59.779716 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.779621    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2" event={"ID":"978eee43-ae26-4500-8236-f39f2c3e123e","Type":"ContainerStarted","Data":"48efb8cec69f0f6f1a34d034ec851921d263e833378501ae1270eebf57e5a91d"}
Sep 10 14:41:59.779716 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.779678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2" event={"ID":"978eee43-ae26-4500-8236-f39f2c3e123e","Type":"ContainerStarted","Data":"2e715ae528db43d17ea7cecec59961fb243e3067ab6db5bf580437c99a6b8e6b"}
Sep 10 14:41:59.779937 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.779795    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:41:59.781222 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.781203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2" event={"ID":"39107e65-6e50-44ff-9943-ad7f8808eb53","Type":"ContainerStarted","Data":"c76c3f37029637ac72eda079a6f732927b5dd572a3f52e420155243ae969710c"}
Sep 10 14:41:59.781316 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.781238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2" event={"ID":"39107e65-6e50-44ff-9943-ad7f8808eb53","Type":"ContainerStarted","Data":"52f074d03a59a941a2460649c5fb1fad64ea107b58732cb32a6f9fde771afcee"}
Sep 10 14:41:59.781316 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.781262    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:41:59.796382 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.796328    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2" podStartSLOduration=1.434565014 podStartE2EDuration="1.79631537s" totalImagesPullingTime="361.750356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:59.794334111 +0000 UTC m=+849.150147041" watchObservedRunningTime="2026-09-10 14:41:59.79631537 +0000 UTC m=+849.152128304"
Sep 10 14:41:59.808672 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:41:59.808634    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2" podStartSLOduration=1.808620163 podStartE2EDuration="1.808620163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:59.807808652 +0000 UTC m=+849.163621586" watchObservedRunningTime="2026-09-10 14:41:59.808620163 +0000 UTC m=+849.164433070"
Sep 10 14:42:00.190987 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:00.190951    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:42:00.440475 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:00.440445    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:42:00.440635 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:00.440489    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:42:00.453593 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:00.453533    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:42:00.798780 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:00.798702    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:42:05.793568 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:05.793537    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:42:10.790849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:10.790813    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:42:16.877372 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:16.877333    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:42:25.847488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:25.847445    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:28.764393 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.764356    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"]
Sep 10 14:42:28.764858 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.764602    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2" podUID="978eee43-ae26-4500-8236-f39f2c3e123e" containerName="maas-api" containerID="cri-o://48efb8cec69f0f6f1a34d034ec851921d263e833378501ae1270eebf57e5a91d" gracePeriod=30
Sep 10 14:42:28.799238 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.799198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:28.827154 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.827117    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"]
Sep 10 14:42:28.827438 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.827397    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2" podUID="39107e65-6e50-44ff-9943-ad7f8808eb53" containerName="payload-pre-processing" containerID="cri-o://c76c3f37029637ac72eda079a6f732927b5dd572a3f52e420155243ae969710c" gracePeriod=30
Sep 10 14:42:28.899868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.899839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:28.964565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.964372    2560 generic.go:356] "Generic (PLEG): container finished" podID="978eee43-ae26-4500-8236-f39f2c3e123e" containerID="48efb8cec69f0f6f1a34d034ec851921d263e833378501ae1270eebf57e5a91d" exitCode=0
Sep 10 14:42:28.964565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.964437    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2" event={"ID":"978eee43-ae26-4500-8236-f39f2c3e123e","Type":"ContainerDied","Data":"48efb8cec69f0f6f1a34d034ec851921d263e833378501ae1270eebf57e5a91d"}
Sep 10 14:42:28.966565 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.966512    2560 generic.go:356] "Generic (PLEG): container finished" podID="39107e65-6e50-44ff-9943-ad7f8808eb53" containerID="c76c3f37029637ac72eda079a6f732927b5dd572a3f52e420155243ae969710c" exitCode=0
Sep 10 14:42:28.966720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.966576    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2" event={"ID":"39107e65-6e50-44ff-9943-ad7f8808eb53","Type":"ContainerDied","Data":"c76c3f37029637ac72eda079a6f732927b5dd572a3f52e420155243ae969710c"}
Sep 10 14:42:28.984559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:28.984521    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:29.045903 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.045873    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:29.107425 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.107401    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:42:29.110333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.110254    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:42:29.190487 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.190452    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-maas-api-tls\") pod \"978eee43-ae26-4500-8236-f39f2c3e123e\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") "
Sep 10 14:42:29.190650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.190496    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-metrics-certs\") pod \"978eee43-ae26-4500-8236-f39f2c3e123e\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") "
Sep 10 14:42:29.190650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.190544    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/978eee43-ae26-4500-8236-f39f2c3e123e-openshift-service-ca\") pod \"978eee43-ae26-4500-8236-f39f2c3e123e\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") "
Sep 10 14:42:29.190650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.190609    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39107e65-6e50-44ff-9943-ad7f8808eb53-plugins-config-volume\") pod \"39107e65-6e50-44ff-9943-ad7f8808eb53\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") "
Sep 10 14:42:29.190650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.190629    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swx7j\" (UniqueName: \"kubernetes.io/projected/978eee43-ae26-4500-8236-f39f2c3e123e-kube-api-access-swx7j\") pod \"978eee43-ae26-4500-8236-f39f2c3e123e\" (UID: \"978eee43-ae26-4500-8236-f39f2c3e123e\") "
Sep 10 14:42:29.190883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.190656    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsxh9\" (UniqueName: \"kubernetes.io/projected/39107e65-6e50-44ff-9943-ad7f8808eb53-kube-api-access-rsxh9\") pod \"39107e65-6e50-44ff-9943-ad7f8808eb53\" (UID: \"39107e65-6e50-44ff-9943-ad7f8808eb53\") "
Sep 10 14:42:29.191067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.191034    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978eee43-ae26-4500-8236-f39f2c3e123e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "978eee43-ae26-4500-8236-f39f2c3e123e" (UID: "978eee43-ae26-4500-8236-f39f2c3e123e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:42:29.191204 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.191042    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39107e65-6e50-44ff-9943-ad7f8808eb53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39107e65-6e50-44ff-9943-ad7f8808eb53" (UID: "39107e65-6e50-44ff-9943-ad7f8808eb53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:42:29.192744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.192715    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "978eee43-ae26-4500-8236-f39f2c3e123e" (UID: "978eee43-ae26-4500-8236-f39f2c3e123e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:29.192829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.192758    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "978eee43-ae26-4500-8236-f39f2c3e123e" (UID: "978eee43-ae26-4500-8236-f39f2c3e123e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:29.192829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.192792    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39107e65-6e50-44ff-9943-ad7f8808eb53-kube-api-access-rsxh9" (OuterVolumeSpecName: "kube-api-access-rsxh9") pod "39107e65-6e50-44ff-9943-ad7f8808eb53" (UID: "39107e65-6e50-44ff-9943-ad7f8808eb53"). InnerVolumeSpecName "kube-api-access-rsxh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:29.192829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.192820    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978eee43-ae26-4500-8236-f39f2c3e123e-kube-api-access-swx7j" (OuterVolumeSpecName: "kube-api-access-swx7j") pod "978eee43-ae26-4500-8236-f39f2c3e123e" (UID: "978eee43-ae26-4500-8236-f39f2c3e123e"). InnerVolumeSpecName "kube-api-access-swx7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:29.291783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.291699    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/978eee43-ae26-4500-8236-f39f2c3e123e-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:42:29.291783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.291731    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39107e65-6e50-44ff-9943-ad7f8808eb53-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:42:29.291783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.291741    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swx7j\" (UniqueName: \"kubernetes.io/projected/978eee43-ae26-4500-8236-f39f2c3e123e-kube-api-access-swx7j\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:42:29.291783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.291751    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsxh9\" (UniqueName: \"kubernetes.io/projected/39107e65-6e50-44ff-9943-ad7f8808eb53-kube-api-access-rsxh9\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:42:29.291783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.291761    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:42:29.291783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.291769    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/978eee43-ae26-4500-8236-f39f2c3e123e-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:42:29.973792 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.973761    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"
Sep 10 14:42:29.974248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.973804    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2" event={"ID":"978eee43-ae26-4500-8236-f39f2c3e123e","Type":"ContainerDied","Data":"2e715ae528db43d17ea7cecec59961fb243e3067ab6db5bf580437c99a6b8e6b"}
Sep 10 14:42:29.974248 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.973878    2560 scope.go:117] "RemoveContainer" containerID="48efb8cec69f0f6f1a34d034ec851921d263e833378501ae1270eebf57e5a91d"
Sep 10 14:42:29.975502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.975473    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2" event={"ID":"39107e65-6e50-44ff-9943-ad7f8808eb53","Type":"ContainerDied","Data":"52f074d03a59a941a2460649c5fb1fad64ea107b58732cb32a6f9fde771afcee"}
Sep 10 14:42:29.975598 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.975500    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"
Sep 10 14:42:29.983693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.983657    2560 scope.go:117] "RemoveContainer" containerID="c76c3f37029637ac72eda079a6f732927b5dd572a3f52e420155243ae969710c"
Sep 10 14:42:29.994072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.994035    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"]
Sep 10 14:42:29.997053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:29.997031    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7b4382a5-55f5fc7584-g98p2"]
Sep 10 14:42:30.005694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:30.005661    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"]
Sep 10 14:42:30.009750 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:30.009732    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7b4382a5-6c48fb96f4-g2ms2"]
Sep 10 14:42:30.132916 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:30.125876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:30.562599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:30.562565    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:30.845560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:30.845467    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:31.144049 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:31.143965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:31.319078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:31.319043    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39107e65-6e50-44ff-9943-ad7f8808eb53" path="/var/lib/kubelet/pods/39107e65-6e50-44ff-9943-ad7f8808eb53/volumes"
Sep 10 14:42:31.319487 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:31.319471    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="978eee43-ae26-4500-8236-f39f2c3e123e" path="/var/lib/kubelet/pods/978eee43-ae26-4500-8236-f39f2c3e123e/volumes"
Sep 10 14:42:33.290653 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:33.290612    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:42:33.350383 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:33.350335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:33.448829 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:33.448798    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:34.593111 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593076    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"]
Sep 10 14:42:34.593510 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593498    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39107e65-6e50-44ff-9943-ad7f8808eb53" containerName="payload-pre-processing"
Sep 10 14:42:34.593510 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593510    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39107e65-6e50-44ff-9943-ad7f8808eb53" containerName="payload-pre-processing"
Sep 10 14:42:34.593587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593530    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="978eee43-ae26-4500-8236-f39f2c3e123e" containerName="maas-api"
Sep 10 14:42:34.593587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593536    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="978eee43-ae26-4500-8236-f39f2c3e123e" containerName="maas-api"
Sep 10 14:42:34.593651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593608    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39107e65-6e50-44ff-9943-ad7f8808eb53" containerName="payload-pre-processing"
Sep 10 14:42:34.593651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.593618    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="978eee43-ae26-4500-8236-f39f2c3e123e" containerName="maas-api"
Sep 10 14:42:34.597883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.597865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.599970 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.599952    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-26c48a75-serving-cert\""
Sep 10 14:42:34.610021 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.609981    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"]
Sep 10 14:42:34.647773 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.647747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:34.744175 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.744141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c17108b-fbeb-4346-91ea-5d40155ad45a-openshift-service-ca\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.744349 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.744195    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-metrics-certs\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.744349 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.744244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7zpw\" (UniqueName: \"kubernetes.io/projected/6c17108b-fbeb-4346-91ea-5d40155ad45a-kube-api-access-j7zpw\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.744457 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.744386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-maas-api-tls\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.761685 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.761659    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:34.845404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.845305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-maas-api-tls\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.845404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.845391    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c17108b-fbeb-4346-91ea-5d40155ad45a-openshift-service-ca\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.845637 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.845423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-metrics-certs\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.845637 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.845440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7zpw\" (UniqueName: \"kubernetes.io/projected/6c17108b-fbeb-4346-91ea-5d40155ad45a-kube-api-access-j7zpw\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.846091 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.846072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c17108b-fbeb-4346-91ea-5d40155ad45a-openshift-service-ca\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.847767 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.847743    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-metrics-certs\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.847897 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.847874    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-maas-api-tls\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.859953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.859925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7zpw\" (UniqueName: \"kubernetes.io/projected/6c17108b-fbeb-4346-91ea-5d40155ad45a-kube-api-access-j7zpw\") pod \"maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:34.907595 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.907569    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:34.912624 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:34.912605    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:35.200261 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.200194    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"]
Sep 10 14:42:35.203207 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:42:35.203180    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c17108b_fbeb_4346_91ea_5d40155ad45a.slice/crio-12fb3fa47b41844834b5606410e832dd687fb265017879ccc2479eb418646e88 WatchSource:0}: Error finding container 12fb3fa47b41844834b5606410e832dd687fb265017879ccc2479eb418646e88: Status 404 returned error can't find the container with id 12fb3fa47b41844834b5606410e832dd687fb265017879ccc2479eb418646e88
Sep 10 14:42:35.252002 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.251958    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:35.372229 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.372189    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:35.452986 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.452909    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"]
Sep 10 14:42:35.457790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.457769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.461156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.461137    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-26c48a75\""
Sep 10 14:42:35.461444 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.461424    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-26c48a75-dockercfg-kkm8f\""
Sep 10 14:42:35.468573 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.468550    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"]
Sep 10 14:42:35.551938 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.551901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf27f6f-ea55-4233-9a2b-644992b02a19-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.552096 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.552005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6w47\" (UniqueName: \"kubernetes.io/projected/fcf27f6f-ea55-4233-9a2b-644992b02a19-kube-api-access-q6w47\") pod \"payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.653467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.653440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf27f6f-ea55-4233-9a2b-644992b02a19-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.653856 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.653523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6w47\" (UniqueName: \"kubernetes.io/projected/fcf27f6f-ea55-4233-9a2b-644992b02a19-kube-api-access-q6w47\") pod \"payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.654053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.654028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf27f6f-ea55-4233-9a2b-644992b02a19-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.662662 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.662635    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6w47\" (UniqueName: \"kubernetes.io/projected/fcf27f6f-ea55-4233-9a2b-644992b02a19-kube-api-access-q6w47\") pod \"payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.770416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.770335    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:35.771418 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:35.771394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:36.019876 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:36.019841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws" event={"ID":"6c17108b-fbeb-4346-91ea-5d40155ad45a","Type":"ContainerStarted","Data":"9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c"}
Sep 10 14:42:36.020030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:36.019894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws" event={"ID":"6c17108b-fbeb-4346-91ea-5d40155ad45a","Type":"ContainerStarted","Data":"12fb3fa47b41844834b5606410e832dd687fb265017879ccc2479eb418646e88"}
Sep 10 14:42:36.020030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:36.020008    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:36.039719 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:36.039615    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws" podStartSLOduration=1.6047129199999999 podStartE2EDuration="2.03959854s" totalImagesPullingTime="434.88562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:36.036500351 +0000 UTC m=+885.392313295" watchObservedRunningTime="2026-09-10 14:42:36.03959854 +0000 UTC m=+885.395411447"
Sep 10 14:42:36.044375 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:36.044347    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"]
Sep 10 14:42:36.046085 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:42:36.046064    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf27f6f_ea55_4233_9a2b_644992b02a19.slice/crio-04b9d1b0460d684e427ba1c7bd898f530884d070225e5c37f29990f24e87cd5c WatchSource:0}: Error finding container 04b9d1b0460d684e427ba1c7bd898f530884d070225e5c37f29990f24e87cd5c: Status 404 returned error can't find the container with id 04b9d1b0460d684e427ba1c7bd898f530884d070225e5c37f29990f24e87cd5c
Sep 10 14:42:36.685665 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:36.685631    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:37.028490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:37.028456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64" event={"ID":"fcf27f6f-ea55-4233-9a2b-644992b02a19","Type":"ContainerStarted","Data":"6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73"}
Sep 10 14:42:37.028677 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:37.028512    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64" event={"ID":"fcf27f6f-ea55-4233-9a2b-644992b02a19","Type":"ContainerStarted","Data":"04b9d1b0460d684e427ba1c7bd898f530884d070225e5c37f29990f24e87cd5c"}
Sep 10 14:42:37.028677 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:37.028550    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:37.045517 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:37.045467    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64" podStartSLOduration=2.045448987 podStartE2EDuration="2.045448987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:37.043738881 +0000 UTC m=+886.399551811" watchObservedRunningTime="2026-09-10 14:42:37.045448987 +0000 UTC m=+886.401261893"
Sep 10 14:42:42.033705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:42.033675    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:42:45.528536 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.528454    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"]
Sep 10 14:42:45.532468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.532447    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.535060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.535037    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5121f4\"/\"kube-root-ca.crt\""
Sep 10 14:42:45.535887 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.535868    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5121f4\"/\"test-model-5121f4-kserve-self-signed-certs\""
Sep 10 14:42:45.535979 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.535952    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5121f4\"/\"default-dockercfg-rk2mx\""
Sep 10 14:42:45.535979 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.535963    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5121f4\"/\"openshift-service-ca.crt\""
Sep 10 14:42:45.547140 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.547116    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"]
Sep 10 14:42:45.552774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.552751    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73a1c90-bfef-4444-ad0e-76239cf818d3-tls-certs\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.552864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.552805    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-tmp-dir\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.552864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.552828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5mdf\" (UniqueName: \"kubernetes.io/projected/d73a1c90-bfef-4444-ad0e-76239cf818d3-kube-api-access-m5mdf\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.552962 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.552890    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-home\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.553028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.553007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-dshm\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.553072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.553056    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-model-cache\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.653776 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.653741    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73a1c90-bfef-4444-ad0e-76239cf818d3-tls-certs\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.653951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.653804    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-tmp-dir\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.653951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.653827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5mdf\" (UniqueName: \"kubernetes.io/projected/d73a1c90-bfef-4444-ad0e-76239cf818d3-kube-api-access-m5mdf\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.653951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.653845    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-home\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.653951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.653888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-dshm\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.653951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.653918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-model-cache\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.654345 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.654321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-tmp-dir\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.654432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.654343    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-home\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.654432 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.654390    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-model-cache\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.656063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.656044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-dshm\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.656369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.656350    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73a1c90-bfef-4444-ad0e-76239cf818d3-tls-certs\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.662446 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.662426    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5mdf\" (UniqueName: \"kubernetes.io/projected/d73a1c90-bfef-4444-ad0e-76239cf818d3-kube-api-access-m5mdf\") pod \"test-model-5121f4-kserve-677d8fcb59-gc2zd\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") " pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:45.799552 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.799449    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:45.842327 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:45.842252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:46.129430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:46.129400    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"]
Sep 10 14:42:46.130861 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:42:46.130833    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd73a1c90_bfef_4444_ad0e_76239cf818d3.slice/crio-7b8052547b41b121a9dc53c039197e4533be0e53ec21fd4d5d70947962ac7989 WatchSource:0}: Error finding container 7b8052547b41b121a9dc53c039197e4533be0e53ec21fd4d5d70947962ac7989: Status 404 returned error can't find the container with id 7b8052547b41b121a9dc53c039197e4533be0e53ec21fd4d5d70947962ac7989
Sep 10 14:42:47.092646 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:47.092609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd" event={"ID":"d73a1c90-bfef-4444-ad0e-76239cf818d3","Type":"ContainerStarted","Data":"c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197"}
Sep 10 14:42:47.093053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:47.092656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd" event={"ID":"d73a1c90-bfef-4444-ad0e-76239cf818d3","Type":"ContainerStarted","Data":"7b8052547b41b121a9dc53c039197e4533be0e53ec21fd4d5d70947962ac7989"}
Sep 10 14:42:47.114680 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:47.114628    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd" podStartSLOduration=2.114611281 podStartE2EDuration="2.114611281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:47.111487422 +0000 UTC m=+896.467300347" watchObservedRunningTime="2026-09-10 14:42:47.114611281 +0000 UTC m=+896.470424189"
Sep 10 14:42:48.039003 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:48.038974    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:42:55.303922 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:55.303893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:55.504442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:55.504409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:55.613673 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:55.613592    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:42:55.842806 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:55.842696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:55.842806 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:55.842736    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:55.866871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:55.855781    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:56.169511 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:56.169424    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:42:56.690332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:56.690272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:58.015706 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:58.015673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:59.314786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:42:59.314758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:05.341499 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.341462    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"]
Sep 10 14:43:05.341859 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.341717    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws" podUID="6c17108b-fbeb-4346-91ea-5d40155ad45a" containerName="maas-api" containerID="cri-o://9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c" gracePeriod=30
Sep 10 14:43:05.411549 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.411518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:05.461738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.461708    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"]
Sep 10 14:43:05.461968 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.461919    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64" podUID="fcf27f6f-ea55-4233-9a2b-644992b02a19" containerName="payload-pre-processing" containerID="cri-o://6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73" gracePeriod=30
Sep 10 14:43:05.540650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.540605    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:05.642700 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.642653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:05.644582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.644559    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:43:05.702219 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.702023    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"]
Sep 10 14:43:05.702492 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.702466    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x" podUID="ec0ee985-2f7b-4545-ae85-c7fd6091cf91" containerName="payload-pre-processing" containerID="cri-o://c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61" gracePeriod=30
Sep 10 14:43:05.745627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.745592    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-metrics-certs\") pod \"6c17108b-fbeb-4346-91ea-5d40155ad45a\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") "
Sep 10 14:43:05.745799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.745644    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-maas-api-tls\") pod \"6c17108b-fbeb-4346-91ea-5d40155ad45a\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") "
Sep 10 14:43:05.745799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.745673    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7zpw\" (UniqueName: \"kubernetes.io/projected/6c17108b-fbeb-4346-91ea-5d40155ad45a-kube-api-access-j7zpw\") pod \"6c17108b-fbeb-4346-91ea-5d40155ad45a\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") "
Sep 10 14:43:05.745799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.745735    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c17108b-fbeb-4346-91ea-5d40155ad45a-openshift-service-ca\") pod \"6c17108b-fbeb-4346-91ea-5d40155ad45a\" (UID: \"6c17108b-fbeb-4346-91ea-5d40155ad45a\") "
Sep 10 14:43:05.746209 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.746174    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c17108b-fbeb-4346-91ea-5d40155ad45a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6c17108b-fbeb-4346-91ea-5d40155ad45a" (UID: "6c17108b-fbeb-4346-91ea-5d40155ad45a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:05.748025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.747997    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6c17108b-fbeb-4346-91ea-5d40155ad45a" (UID: "6c17108b-fbeb-4346-91ea-5d40155ad45a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:05.748212 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.748195    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6c17108b-fbeb-4346-91ea-5d40155ad45a" (UID: "6c17108b-fbeb-4346-91ea-5d40155ad45a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:05.748271 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.748243    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c17108b-fbeb-4346-91ea-5d40155ad45a-kube-api-access-j7zpw" (OuterVolumeSpecName: "kube-api-access-j7zpw") pod "6c17108b-fbeb-4346-91ea-5d40155ad45a" (UID: "6c17108b-fbeb-4346-91ea-5d40155ad45a"). InnerVolumeSpecName "kube-api-access-j7zpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:05.795079 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.795049    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:05.817660 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.817576    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:43:05.846951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.846911    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c17108b-fbeb-4346-91ea-5d40155ad45a-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:05.846951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.846945    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:05.846951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.846961    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c17108b-fbeb-4346-91ea-5d40155ad45a-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:05.847344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.846972    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7zpw\" (UniqueName: \"kubernetes.io/projected/6c17108b-fbeb-4346-91ea-5d40155ad45a-kube-api-access-j7zpw\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:05.907758 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.907730    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:05.948117 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.948001    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf27f6f-ea55-4233-9a2b-644992b02a19-plugins-config-volume\") pod \"fcf27f6f-ea55-4233-9a2b-644992b02a19\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") "
Sep 10 14:43:05.948364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.948345    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6w47\" (UniqueName: \"kubernetes.io/projected/fcf27f6f-ea55-4233-9a2b-644992b02a19-kube-api-access-q6w47\") pod \"fcf27f6f-ea55-4233-9a2b-644992b02a19\" (UID: \"fcf27f6f-ea55-4233-9a2b-644992b02a19\") "
Sep 10 14:43:05.948504 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.948466    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf27f6f-ea55-4233-9a2b-644992b02a19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fcf27f6f-ea55-4233-9a2b-644992b02a19" (UID: "fcf27f6f-ea55-4233-9a2b-644992b02a19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:05.948743 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.948725    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf27f6f-ea55-4233-9a2b-644992b02a19-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:05.951354 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.951331    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf27f6f-ea55-4233-9a2b-644992b02a19-kube-api-access-q6w47" (OuterVolumeSpecName: "kube-api-access-q6w47") pod "fcf27f6f-ea55-4233-9a2b-644992b02a19" (UID: "fcf27f6f-ea55-4233-9a2b-644992b02a19"). InnerVolumeSpecName "kube-api-access-q6w47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:05.975887 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:05.975857    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:43:06.019211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.019163    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:06.049521 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.049485    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-plugins-config-volume\") pod \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") "
Sep 10 14:43:06.049693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.049540    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m66r\" (UniqueName: \"kubernetes.io/projected/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-kube-api-access-7m66r\") pod \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\" (UID: \"ec0ee985-2f7b-4545-ae85-c7fd6091cf91\") "
Sep 10 14:43:06.049781 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.049768    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6w47\" (UniqueName: \"kubernetes.io/projected/fcf27f6f-ea55-4233-9a2b-644992b02a19-kube-api-access-q6w47\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:06.049971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.049943    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec0ee985-2f7b-4545-ae85-c7fd6091cf91" (UID: "ec0ee985-2f7b-4545-ae85-c7fd6091cf91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:06.051633 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.051615    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-kube-api-access-7m66r" (OuterVolumeSpecName: "kube-api-access-7m66r") pod "ec0ee985-2f7b-4545-ae85-c7fd6091cf91" (UID: "ec0ee985-2f7b-4545-ae85-c7fd6091cf91"). InnerVolumeSpecName "kube-api-access-7m66r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:06.134333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.134301    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:06.150784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.150716    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:06.150784 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.150740    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m66r\" (UniqueName: \"kubernetes.io/projected/ec0ee985-2f7b-4545-ae85-c7fd6091cf91-kube-api-access-7m66r\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:43:06.217588 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.217549    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec0ee985-2f7b-4545-ae85-c7fd6091cf91" containerID="c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61" exitCode=0
Sep 10 14:43:06.217588 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.217592    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"
Sep 10 14:43:06.217842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.217646    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x" event={"ID":"ec0ee985-2f7b-4545-ae85-c7fd6091cf91","Type":"ContainerDied","Data":"c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61"}
Sep 10 14:43:06.217842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.217706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x" event={"ID":"ec0ee985-2f7b-4545-ae85-c7fd6091cf91","Type":"ContainerDied","Data":"647db71f8ddaf534861fbc44b6fd2f5f4043ac137352eaaea5920918ba235c9b"}
Sep 10 14:43:06.217842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.217735    2560 scope.go:117] "RemoveContainer" containerID="c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61"
Sep 10 14:43:06.219296 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.219259    2560 generic.go:356] "Generic (PLEG): container finished" podID="fcf27f6f-ea55-4233-9a2b-644992b02a19" containerID="6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73" exitCode=0
Sep 10 14:43:06.219411 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.219321    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64" event={"ID":"fcf27f6f-ea55-4233-9a2b-644992b02a19","Type":"ContainerDied","Data":"6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73"}
Sep 10 14:43:06.219411 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.219372    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64" event={"ID":"fcf27f6f-ea55-4233-9a2b-644992b02a19","Type":"ContainerDied","Data":"04b9d1b0460d684e427ba1c7bd898f530884d070225e5c37f29990f24e87cd5c"}
Sep 10 14:43:06.219529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.219407    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"
Sep 10 14:43:06.220740 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.220716    2560 generic.go:356] "Generic (PLEG): container finished" podID="6c17108b-fbeb-4346-91ea-5d40155ad45a" containerID="9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c" exitCode=0
Sep 10 14:43:06.220853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.220752    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"
Sep 10 14:43:06.220853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.220771    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws" event={"ID":"6c17108b-fbeb-4346-91ea-5d40155ad45a","Type":"ContainerDied","Data":"9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c"}
Sep 10 14:43:06.220853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.220808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws" event={"ID":"6c17108b-fbeb-4346-91ea-5d40155ad45a","Type":"ContainerDied","Data":"12fb3fa47b41844834b5606410e832dd687fb265017879ccc2479eb418646e88"}
Sep 10 14:43:06.229818 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.229772    2560 scope.go:117] "RemoveContainer" containerID="c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61"
Sep 10 14:43:06.230156 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:43:06.230130    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61\": container with ID starting with c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61 not found: ID does not exist" containerID="c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61"
Sep 10 14:43:06.230237 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.230165    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61"} err="failed to get container status \"c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61\": rpc error: code = NotFound desc = could not find container \"c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61\": container with ID starting with c8df7de2310b4aee5fd73f783461bbf52e18ed65c227c1f11885915c4c9f7e61 not found: ID does not exist"
Sep 10 14:43:06.230237 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.230182    2560 scope.go:117] "RemoveContainer" containerID="6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73"
Sep 10 14:43:06.238841 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.238817    2560 scope.go:117] "RemoveContainer" containerID="6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73"
Sep 10 14:43:06.239156 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:43:06.239136    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73\": container with ID starting with 6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73 not found: ID does not exist" containerID="6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73"
Sep 10 14:43:06.239231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.239166    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73"} err="failed to get container status \"6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73\": rpc error: code = NotFound desc = could not find container \"6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73\": container with ID starting with 6acd8782e20bf2958a61b09f291102a9903935b590b225775f2d471a9243ea73 not found: ID does not exist"
Sep 10 14:43:06.239231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.239196    2560 scope.go:117] "RemoveContainer" containerID="9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c"
Sep 10 14:43:06.245118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.245098    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"]
Sep 10 14:43:06.248077 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.248059    2560 scope.go:117] "RemoveContainer" containerID="9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c"
Sep 10 14:43:06.248356 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:43:06.248336    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c\": container with ID starting with 9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c not found: ID does not exist" containerID="9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c"
Sep 10 14:43:06.248412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.248364    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c"} err="failed to get container status \"9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c\": rpc error: code = NotFound desc = could not find container \"9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c\": container with ID starting with 9b65453d0157f1a82b8766e7ca92ccb4c11069896d6bc1fec660ad231d8a786c not found: ID does not exist"
Sep 10 14:43:06.253196 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.253177    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22fb28-869d44786-bj89x"]
Sep 10 14:43:06.267858 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.267832    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"]
Sep 10 14:43:06.278429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.278404    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-26c48a75-68c7d57ff-d44ws"]
Sep 10 14:43:06.297554 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.297527    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"]
Sep 10 14:43:06.309235 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.309208    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-26c48a75-54f56988b6-gzq64"]
Sep 10 14:43:06.478710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.478678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:06.745207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.745126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:06.945663 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:06.945629    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:07.318876 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:07.318835    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c17108b-fbeb-4346-91ea-5d40155ad45a" path="/var/lib/kubelet/pods/6c17108b-fbeb-4346-91ea-5d40155ad45a/volumes"
Sep 10 14:43:07.319334 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:07.319313    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec0ee985-2f7b-4545-ae85-c7fd6091cf91" path="/var/lib/kubelet/pods/ec0ee985-2f7b-4545-ae85-c7fd6091cf91/volumes"
Sep 10 14:43:07.319694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:07.319677    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf27f6f-ea55-4233-9a2b-644992b02a19" path="/var/lib/kubelet/pods/fcf27f6f-ea55-4233-9a2b-644992b02a19/volumes"
Sep 10 14:43:08.405705 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:08.405667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:13.958190 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:13.958155    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:14.060319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:14.060247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:14.241385 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:14.241306    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:43:14.257786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:14.257754    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:14.753194 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:14.753158    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:15.252494 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:15.252464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:15.408471 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:15.408440    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:16.848807 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:16.848769    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:16.964292 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:16.964255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:17.154555 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:17.154467    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:17.253459 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:17.253431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:17.625499 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:17.625463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:18.578491 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:18.578445    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:18.718686 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:18.718658    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:20.031384 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:20.031352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:20.274501 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:20.274453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:20.829255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:20.829217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:20.931473 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:20.931443    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:27.642788 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.642743    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"]
Sep 10 14:43:27.643674 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643652    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c17108b-fbeb-4346-91ea-5d40155ad45a" containerName="maas-api"
Sep 10 14:43:27.643736 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643679    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c17108b-fbeb-4346-91ea-5d40155ad45a" containerName="maas-api"
Sep 10 14:43:27.643736 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643705    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf27f6f-ea55-4233-9a2b-644992b02a19" containerName="payload-pre-processing"
Sep 10 14:43:27.643736 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643714    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf27f6f-ea55-4233-9a2b-644992b02a19" containerName="payload-pre-processing"
Sep 10 14:43:27.643830 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643739    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec0ee985-2f7b-4545-ae85-c7fd6091cf91" containerName="payload-pre-processing"
Sep 10 14:43:27.643830 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643748    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec0ee985-2f7b-4545-ae85-c7fd6091cf91" containerName="payload-pre-processing"
Sep 10 14:43:27.643947 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643933    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec0ee985-2f7b-4545-ae85-c7fd6091cf91" containerName="payload-pre-processing"
Sep 10 14:43:27.643983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643959    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf27f6f-ea55-4233-9a2b-644992b02a19" containerName="payload-pre-processing"
Sep 10 14:43:27.643983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.643976    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c17108b-fbeb-4346-91ea-5d40155ad45a" containerName="maas-api"
Sep 10 14:43:27.648789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.648766    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.651495 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.651470    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5b0323\"/\"premium-w4-5b0323-kserve-self-signed-certs\""
Sep 10 14:43:27.651645 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.651511    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-5b0323\"/\"kube-root-ca.crt\""
Sep 10 14:43:27.651756 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.651731    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-5b0323\"/\"openshift-service-ca.crt\""
Sep 10 14:43:27.651957 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.651908    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5b0323\"/\"default-dockercfg-r4d7f\""
Sep 10 14:43:27.654259 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.654241    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"]
Sep 10 14:43:27.659844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.659821    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tls-certs\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.659958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.659849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tmp-dir\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.659958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.659879    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-model-cache\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.659958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.659909    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcbw7\" (UniqueName: \"kubernetes.io/projected/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-kube-api-access-rcbw7\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.659958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.659932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-dshm\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.660181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.659973    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-home\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.739428 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.739400    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:27.760718 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.760685    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-home\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.760853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.760739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tls-certs\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.760853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.760757    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tmp-dir\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.760853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.760785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-model-cache\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.760853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.760811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcbw7\" (UniqueName: \"kubernetes.io/projected/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-kube-api-access-rcbw7\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.760853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.760828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-dshm\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.761169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.761142    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-home\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.761233 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.761154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tmp-dir\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.761233 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.761184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-model-cache\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.763086 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.763062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-dshm\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.763242 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.763225    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tls-certs\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.768544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.768524    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcbw7\" (UniqueName: \"kubernetes.io/projected/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-kube-api-access-rcbw7\") pod \"premium-w4-5b0323-kserve-6449d484b7-9wbs8\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") " pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:27.959865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:27.959822    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:28.168524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:28.168487    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:43:28.246524 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:28.246501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"]
Sep 10 14:43:28.249063 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:43:28.249034    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ebd05e7_18d6_4005_ac8f_77313ccaaa25.slice/crio-2f5ea2114fda6617428adc9fbee3214c410181dd44ece91323940959c07413f4 WatchSource:0}: Error finding container 2f5ea2114fda6617428adc9fbee3214c410181dd44ece91323940959c07413f4: Status 404 returned error can't find the container with id 2f5ea2114fda6617428adc9fbee3214c410181dd44ece91323940959c07413f4
Sep 10 14:43:28.360502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:28.360466    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8" event={"ID":"4ebd05e7-18d6-4005-ac8f-77313ccaaa25","Type":"ContainerStarted","Data":"f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa"}
Sep 10 14:43:28.360665 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:28.360518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8" event={"ID":"4ebd05e7-18d6-4005-ac8f-77313ccaaa25","Type":"ContainerStarted","Data":"2f5ea2114fda6617428adc9fbee3214c410181dd44ece91323940959c07413f4"}
Sep 10 14:43:28.397507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:28.397442    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8" podStartSLOduration=1.397423296 podStartE2EDuration="1.397423296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:28.394873914 +0000 UTC m=+937.750686820" watchObservedRunningTime="2026-09-10 14:43:28.397423296 +0000 UTC m=+937.753236207"
Sep 10 14:43:37.960275 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:37.960238    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:37.960838 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:37.960306    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:37.972811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:37.972788    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:38.434499 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:38.434418    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:43:43.660817 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:43.660782    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:43.768119 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:43.768089    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:45.729713 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:45.729653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.217253 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:49.217222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.290177 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:49.290144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.382721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:49.382676    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:50.288996 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:50.288958    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:43:51.020251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.020216    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"]
Sep 10 14:43:51.030935 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.030785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.033229 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.033204    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"test-model-2dccd5-kserve-self-signed-certs\""
Sep 10 14:43:51.033814 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.033789    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"kube-root-ca.crt\""
Sep 10 14:43:51.033938 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.033829    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"openshift-service-ca.crt\""
Sep 10 14:43:51.033938 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.033789    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"default-dockercfg-8xblg\""
Sep 10 14:43:51.034895 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.034874    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"]
Sep 10 14:43:51.050772 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.050742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:51.084071 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.084038    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-model-cache\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.084226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.084080    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tmp-dir\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.084226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.084109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-home\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.084333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.084222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbf9l\" (UniqueName: \"kubernetes.io/projected/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-kube-api-access-nbf9l\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.084333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.084250    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tls-certs\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.084333 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.084312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-dshm\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.184944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.184906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbf9l\" (UniqueName: \"kubernetes.io/projected/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-kube-api-access-nbf9l\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.184944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.184944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tls-certs\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.184969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-dshm\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.185033    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-model-cache\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.185071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tmp-dir\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.185102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-home\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.185506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-model-cache\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.185522    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tmp-dir\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.185625 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.185555    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-home\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.187327 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.187309    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-dshm\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.189472 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.189456    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"test-model-2dccd5-kserve-self-signed-certs\""
Sep 10 14:43:51.197678 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.197654    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tls-certs\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.197842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.197826    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"kube-root-ca.crt\""
Sep 10 14:43:51.208885 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.208860    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"openshift-service-ca.crt\""
Sep 10 14:43:51.219426 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.219396    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbf9l\" (UniqueName: \"kubernetes.io/projected/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-kube-api-access-nbf9l\") pod \"test-model-2dccd5-kserve-6bf74f4787-2fmpr\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") " pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.346647 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.346574    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2dccd5\"/\"default-dockercfg-8xblg\""
Sep 10 14:43:51.355341 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.355319    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:43:51.633671 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:51.633643    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"]
Sep 10 14:43:51.634640 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:43:51.634578    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bccdbc5_8409_4ea5_8ff9_48743297a5b9.slice/crio-fa797ffb7e813a8b038c76444a567bd13c8c4f1dfcf677760f5984a83a81a3ba WatchSource:0}: Error finding container fa797ffb7e813a8b038c76444a567bd13c8c4f1dfcf677760f5984a83a81a3ba: Status 404 returned error can't find the container with id fa797ffb7e813a8b038c76444a567bd13c8c4f1dfcf677760f5984a83a81a3ba
Sep 10 14:43:52.508317 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:52.508262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr" event={"ID":"8bccdbc5-8409-4ea5-8ff9-48743297a5b9","Type":"ContainerStarted","Data":"e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81"}
Sep 10 14:43:52.508690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:52.508330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr" event={"ID":"8bccdbc5-8409-4ea5-8ff9-48743297a5b9","Type":"ContainerStarted","Data":"fa797ffb7e813a8b038c76444a567bd13c8c4f1dfcf677760f5984a83a81a3ba"}
Sep 10 14:43:52.526945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:43:52.526893    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr" podStartSLOduration=1.526878447 podStartE2EDuration="1.526878447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:52.525432978 +0000 UTC m=+961.881245887" watchObservedRunningTime="2026-09-10 14:43:52.526878447 +0000 UTC m=+961.882691353"
Sep 10 14:44:01.355990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:01.355945    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:44:01.355990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:01.355995    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:44:01.369112 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:01.369081    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:44:01.573221 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:01.573193    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:44:04.770744 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:04.770715    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:05.289309 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:05.289266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:11.155296 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:11.155253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:11.270128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:11.270097    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:11.438966 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:11.438932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:12.889449 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:12.889417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:44:21.848611 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:21.848526    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:22.628911 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:22.628874    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:26.624603 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:26.624556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:28.686425 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:28.686374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:28.756720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:28.756695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:28.856315 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:28.856268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:28.955264 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:28.955187    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:29.662335 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:29.662306    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:29.749142 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:29.749111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:29.849779 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:29.849748    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:31.317061 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:31.317026    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:37.134295 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:37.134241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:37.233640 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:37.233608    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:37.819824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:37.819789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:39.885631 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:39.885594    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:44:44.432667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:44.432628    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:44.502509 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:44.502477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:44.676917 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:44.676883    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:44.989042 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:44.989009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:45.948078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:45.948039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:50.729835 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:50.729768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:50.832018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:50.831979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:50.930720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:50.930691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:51.044896 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:51.044815    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:51.147455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:51.147422    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:51.242460 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:51.242421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:51.346107 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:51.346023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:52.442220 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:52.442180    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:52.546718 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:52.546683    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:52.776173 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:52.776060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:52.928497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:52.928464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:53.028991 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.028907    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:53.111208 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.111177    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"]
Sep 10 14:44:53.114796 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.114779    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.117289 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.117257    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-785829b5-dockercfg-sm4rp\""
Sep 10 14:44:53.117404 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.117320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-785829b5\""
Sep 10 14:44:53.126059 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.126033    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"]
Sep 10 14:44:53.202925 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.202886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d583ab7-0fe1-4930-93e3-644c2f708fc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.203092 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.202933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxgtx\" (UniqueName: \"kubernetes.io/projected/2d583ab7-0fe1-4930-93e3-644c2f708fc2-kube-api-access-lxgtx\") pod \"payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.303865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.303775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d583ab7-0fe1-4930-93e3-644c2f708fc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.303865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.303842    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxgtx\" (UniqueName: \"kubernetes.io/projected/2d583ab7-0fe1-4930-93e3-644c2f708fc2-kube-api-access-lxgtx\") pod \"payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.304483 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.304463    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d583ab7-0fe1-4930-93e3-644c2f708fc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.311727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.311693    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxgtx\" (UniqueName: \"kubernetes.io/projected/2d583ab7-0fe1-4930-93e3-644c2f708fc2-kube-api-access-lxgtx\") pod \"payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.425638 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.425600    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.702893 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.702865    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"]
Sep 10 14:44:53.704234 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:44:53.704206    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d583ab7_0fe1_4930_93e3_644c2f708fc2.slice/crio-e307f0c7a23593e78aa33a1ac6ff4029761c15b90b2c104674d6b138d741009e WatchSource:0}: Error finding container e307f0c7a23593e78aa33a1ac6ff4029761c15b90b2c104674d6b138d741009e: Status 404 returned error can't find the container with id e307f0c7a23593e78aa33a1ac6ff4029761c15b90b2c104674d6b138d741009e
Sep 10 14:44:53.818742 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.818710    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:53.877871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.877835    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r" event={"ID":"2d583ab7-0fe1-4930-93e3-644c2f708fc2","Type":"ContainerStarted","Data":"00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db"}
Sep 10 14:44:53.878052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.877895    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r" event={"ID":"2d583ab7-0fe1-4930-93e3-644c2f708fc2","Type":"ContainerStarted","Data":"e307f0c7a23593e78aa33a1ac6ff4029761c15b90b2c104674d6b138d741009e"}
Sep 10 14:44:53.878052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.877983    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:44:53.899453 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:53.899388    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r" podStartSLOduration=0.899367477 podStartE2EDuration="899.367477ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:53.894121359 +0000 UTC m=+1023.249934265" watchObservedRunningTime="2026-09-10 14:44:53.899367477 +0000 UTC m=+1023.255180385"
Sep 10 14:44:56.402499 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:56.402472    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:58.730580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:44:58.730549    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:04.885796 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:04.885762    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:45:08.188614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:08.188561    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:45:11.624425 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:11.624393    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:16.068146 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:16.068111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:16.720506 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:16.720470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:16.817654 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:16.817584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:17.212255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:17.212221    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:23.453252 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:23.453218    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:23.551738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:23.551700    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:32.609536 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:32.609499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:33.061691 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:33.061655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:33.305344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:33.305313    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:33.738774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:33.738744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:34.450060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.450025    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"]
Sep 10 14:45:34.450324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.450297    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r" podUID="2d583ab7-0fe1-4930-93e3-644c2f708fc2" containerName="payload-pre-processing" containerID="cri-o://00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db" gracePeriod=30
Sep 10 14:45:34.713792 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.713733    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:45:34.811670 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.811637    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d583ab7-0fe1-4930-93e3-644c2f708fc2-plugins-config-volume\") pod \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") "
Sep 10 14:45:34.812147 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.811689    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxgtx\" (UniqueName: \"kubernetes.io/projected/2d583ab7-0fe1-4930-93e3-644c2f708fc2-kube-api-access-lxgtx\") pod \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\" (UID: \"2d583ab7-0fe1-4930-93e3-644c2f708fc2\") "
Sep 10 14:45:34.812147 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.812102    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d583ab7-0fe1-4930-93e3-644c2f708fc2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d583ab7-0fe1-4930-93e3-644c2f708fc2" (UID: "2d583ab7-0fe1-4930-93e3-644c2f708fc2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:45:34.813809 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.813779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d583ab7-0fe1-4930-93e3-644c2f708fc2-kube-api-access-lxgtx" (OuterVolumeSpecName: "kube-api-access-lxgtx") pod "2d583ab7-0fe1-4930-93e3-644c2f708fc2" (UID: "2d583ab7-0fe1-4930-93e3-644c2f708fc2"). InnerVolumeSpecName "kube-api-access-lxgtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:45:34.912684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.912647    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d583ab7-0fe1-4930-93e3-644c2f708fc2-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:34.912684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:34.912678    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxgtx\" (UniqueName: \"kubernetes.io/projected/2d583ab7-0fe1-4930-93e3-644c2f708fc2-kube-api-access-lxgtx\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:35.012773 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.012685    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:35.113202 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.113158    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:35.116899 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.116873    2560 generic.go:356] "Generic (PLEG): container finished" podID="2d583ab7-0fe1-4930-93e3-644c2f708fc2" containerID="00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db" exitCode=0
Sep 10 14:45:35.117031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.116917    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"
Sep 10 14:45:35.117031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.116913    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r" event={"ID":"2d583ab7-0fe1-4930-93e3-644c2f708fc2","Type":"ContainerDied","Data":"00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db"}
Sep 10 14:45:35.117031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.116981    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r" event={"ID":"2d583ab7-0fe1-4930-93e3-644c2f708fc2","Type":"ContainerDied","Data":"e307f0c7a23593e78aa33a1ac6ff4029761c15b90b2c104674d6b138d741009e"}
Sep 10 14:45:35.117031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.117013    2560 scope.go:117] "RemoveContainer" containerID="00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db"
Sep 10 14:45:35.126693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.126674    2560 scope.go:117] "RemoveContainer" containerID="00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db"
Sep 10 14:45:35.127003 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:45:35.126979    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db\": container with ID starting with 00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db not found: ID does not exist" containerID="00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db"
Sep 10 14:45:35.127082 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.127010    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db"} err="failed to get container status \"00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db\": rpc error: code = NotFound desc = could not find container \"00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db\": container with ID starting with 00cbd4372036a5fa3ec7f2b343a6970f19362935907bc65e6b9626af75def3db not found: ID does not exist"
Sep 10 14:45:35.141584 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.141561    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"]
Sep 10 14:45:35.147293 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.146597    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-785829b5-6dfb974db-nwg2r"]
Sep 10 14:45:35.287910 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.287829    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:45:35.319430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.319400    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d583ab7-0fe1-4930-93e3-644c2f708fc2" path="/var/lib/kubelet/pods/2d583ab7-0fe1-4930-93e3-644c2f708fc2/volumes"
Sep 10 14:45:35.630599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.630524    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:35.729056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.729027    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:35.842112 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:35.842079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:40.619651 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:40.619616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:40.723056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:40.723022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:40.859655 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:40.859625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:41.203486 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:41.203450    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:41.325095 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:41.325058    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:41.333655 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:41.333623    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"]
Sep 10 14:45:41.403799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:41.403769    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:47.399587 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:47.399506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:47.499049 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:47.499021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:47.600140 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:47.600103    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:51.877533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:51.877500    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:53.183442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:53.183407    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:53.185125 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:53.185104    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:45:53.582403 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:53.582368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:53.649958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:53.649931    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:54.496580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:54.496539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:54.805025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:54.804940    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:54.904022 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:54.903990    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:55.280988 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:55.280956    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:56.353675 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.353631    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd" podUID="d73a1c90-bfef-4444-ad0e-76239cf818d3" containerName="main" containerID="cri-o://c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197" gracePeriod=45
Sep 10 14:45:56.620558 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.620532    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:45:56.727881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.727846    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73a1c90-bfef-4444-ad0e-76239cf818d3-tls-certs\") pod \"d73a1c90-bfef-4444-ad0e-76239cf818d3\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") "
Sep 10 14:45:56.728050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.727945    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-dshm\") pod \"d73a1c90-bfef-4444-ad0e-76239cf818d3\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") "
Sep 10 14:45:56.728050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.727976    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5mdf\" (UniqueName: \"kubernetes.io/projected/d73a1c90-bfef-4444-ad0e-76239cf818d3-kube-api-access-m5mdf\") pod \"d73a1c90-bfef-4444-ad0e-76239cf818d3\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") "
Sep 10 14:45:56.728050 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728005    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-home\") pod \"d73a1c90-bfef-4444-ad0e-76239cf818d3\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") "
Sep 10 14:45:56.728199 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728065    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-model-cache\") pod \"d73a1c90-bfef-4444-ad0e-76239cf818d3\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") "
Sep 10 14:45:56.728199 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728141    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-tmp-dir\") pod \"d73a1c90-bfef-4444-ad0e-76239cf818d3\" (UID: \"d73a1c90-bfef-4444-ad0e-76239cf818d3\") "
Sep 10 14:45:56.728435 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728391    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-home" (OuterVolumeSpecName: "home") pod "d73a1c90-bfef-4444-ad0e-76239cf818d3" (UID: "d73a1c90-bfef-4444-ad0e-76239cf818d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:45:56.728435 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728399    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "d73a1c90-bfef-4444-ad0e-76239cf818d3" (UID: "d73a1c90-bfef-4444-ad0e-76239cf818d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:45:56.728590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728537    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:56.728590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728536    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d73a1c90-bfef-4444-ad0e-76239cf818d3" (UID: "d73a1c90-bfef-4444-ad0e-76239cf818d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:45:56.728590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.728558    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:56.730318 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.730287    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d73a1c90-bfef-4444-ad0e-76239cf818d3-kube-api-access-m5mdf" (OuterVolumeSpecName: "kube-api-access-m5mdf") pod "d73a1c90-bfef-4444-ad0e-76239cf818d3" (UID: "d73a1c90-bfef-4444-ad0e-76239cf818d3"). InnerVolumeSpecName "kube-api-access-m5mdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:45:56.730318 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.730297    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-dshm" (OuterVolumeSpecName: "dshm") pod "d73a1c90-bfef-4444-ad0e-76239cf818d3" (UID: "d73a1c90-bfef-4444-ad0e-76239cf818d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:45:56.730434 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.730333    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d73a1c90-bfef-4444-ad0e-76239cf818d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d73a1c90-bfef-4444-ad0e-76239cf818d3" (UID: "d73a1c90-bfef-4444-ad0e-76239cf818d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:45:56.829324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.829264    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:56.829324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.829319    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5mdf\" (UniqueName: \"kubernetes.io/projected/d73a1c90-bfef-4444-ad0e-76239cf818d3-kube-api-access-m5mdf\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:56.829324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.829330    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73a1c90-bfef-4444-ad0e-76239cf818d3-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:56.829556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:56.829339    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73a1c90-bfef-4444-ad0e-76239cf818d3-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:57.259891 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.259852    2560 generic.go:356] "Generic (PLEG): container finished" podID="d73a1c90-bfef-4444-ad0e-76239cf818d3" containerID="c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197" exitCode=0
Sep 10 14:45:57.260060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.259902    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"
Sep 10 14:45:57.260060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.259911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd" event={"ID":"d73a1c90-bfef-4444-ad0e-76239cf818d3","Type":"ContainerDied","Data":"c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197"}
Sep 10 14:45:57.260060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.259949    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd" event={"ID":"d73a1c90-bfef-4444-ad0e-76239cf818d3","Type":"ContainerDied","Data":"7b8052547b41b121a9dc53c039197e4533be0e53ec21fd4d5d70947962ac7989"}
Sep 10 14:45:57.260060 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.259970    2560 scope.go:117] "RemoveContainer" containerID="c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197"
Sep 10 14:45:57.269728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.269710    2560 scope.go:117] "RemoveContainer" containerID="c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197"
Sep 10 14:45:57.269996 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:45:57.269979    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197\": container with ID starting with c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197 not found: ID does not exist" containerID="c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197"
Sep 10 14:45:57.270064 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.270005    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197"} err="failed to get container status \"c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197\": rpc error: code = NotFound desc = could not find container \"c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197\": container with ID starting with c4bdd285ed1e13d9672300aed0837016d1bdb4453d5612bf4e4c46bdf3c00197 not found: ID does not exist"
Sep 10 14:45:57.281360 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.281332    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"]
Sep 10 14:45:57.284724 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.284701    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5121f4/test-model-5121f4-kserve-677d8fcb59-gc2zd"]
Sep 10 14:45:57.318785 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.318753    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d73a1c90-bfef-4444-ad0e-76239cf818d3" path="/var/lib/kubelet/pods/d73a1c90-bfef-4444-ad0e-76239cf818d3/volumes"
Sep 10 14:45:57.545982 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.545896    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:57.645783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:57.645749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:58.578582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:58.578550    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:58.652832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:58.652793    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:58.753646 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:58.753616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:59.075990 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.075952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:59.174231 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.174200    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:59.210690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.210654    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"]
Sep 10 14:45:59.211130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.211114    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73a1c90-bfef-4444-ad0e-76239cf818d3" containerName="main"
Sep 10 14:45:59.211188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.211132    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73a1c90-bfef-4444-ad0e-76239cf818d3" containerName="main"
Sep 10 14:45:59.211188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.211146    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d583ab7-0fe1-4930-93e3-644c2f708fc2" containerName="payload-pre-processing"
Sep 10 14:45:59.211188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.211161    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d583ab7-0fe1-4930-93e3-644c2f708fc2" containerName="payload-pre-processing"
Sep 10 14:45:59.211300 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.211233    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73a1c90-bfef-4444-ad0e-76239cf818d3" containerName="main"
Sep 10 14:45:59.211300 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.211241    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d583ab7-0fe1-4930-93e3-644c2f708fc2" containerName="payload-pre-processing"
Sep 10 14:45:59.215424 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.215405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.217637 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.217618    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-5c9641-dockercfg-r79vb\""
Sep 10 14:45:59.217740 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.217616    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-5c9641\""
Sep 10 14:45:59.225840 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.225817    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"]
Sep 10 14:45:59.256589 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.256556    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"]
Sep 10 14:45:59.260120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.260102    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.269390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.269363    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"]
Sep 10 14:45:59.354952 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.354870    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2be7141-d1bb-4a32-9218-f36497842ad5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.354952 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.354928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r86cm\" (UniqueName: \"kubernetes.io/projected/41064590-4693-429b-9892-eefc37ce1a68-kube-api-access-r86cm\") pod \"payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.355158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.354984    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41064590-4693-429b-9892-eefc37ce1a68-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.355158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.355048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lndtp\" (UniqueName: \"kubernetes.io/projected/f2be7141-d1bb-4a32-9218-f36497842ad5-kube-api-access-lndtp\") pod \"payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.455961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.455927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41064590-4693-429b-9892-eefc37ce1a68-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.456128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.455986    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lndtp\" (UniqueName: \"kubernetes.io/projected/f2be7141-d1bb-4a32-9218-f36497842ad5-kube-api-access-lndtp\") pod \"payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.456128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.456022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2be7141-d1bb-4a32-9218-f36497842ad5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.456128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.456051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r86cm\" (UniqueName: \"kubernetes.io/projected/41064590-4693-429b-9892-eefc37ce1a68-kube-api-access-r86cm\") pod \"payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.456738 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.456713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41064590-4693-429b-9892-eefc37ce1a68-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.456826 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.456768    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2be7141-d1bb-4a32-9218-f36497842ad5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.464340 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.464317    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r86cm\" (UniqueName: \"kubernetes.io/projected/41064590-4693-429b-9892-eefc37ce1a68-kube-api-access-r86cm\") pod \"payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.464496 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.464480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lndtp\" (UniqueName: \"kubernetes.io/projected/f2be7141-d1bb-4a32-9218-f36497842ad5-kube-api-access-lndtp\") pod \"payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.527184 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.527156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:45:59.571392 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.571357    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:45:59.804119 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.804084    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"]
Sep 10 14:45:59.805370 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:45:59.805347    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2be7141_d1bb_4a32_9218_f36497842ad5.slice/crio-39a12e67c2e1737930ae1480dca941cc5f3c6e63cbbfeed38c463e6b5f2d56cc WatchSource:0}: Error finding container 39a12e67c2e1737930ae1480dca941cc5f3c6e63cbbfeed38c463e6b5f2d56cc: Status 404 returned error can't find the container with id 39a12e67c2e1737930ae1480dca941cc5f3c6e63cbbfeed38c463e6b5f2d56cc
Sep 10 14:45:59.860572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:45:59.860549    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"]
Sep 10 14:45:59.862308 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:45:59.862261    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41064590_4693_429b_9892_eefc37ce1a68.slice/crio-86c53154c9a358d131a3fbc41da32cdf5c8d45b8aea2855d24b98dd93d0bd436 WatchSource:0}: Error finding container 86c53154c9a358d131a3fbc41da32cdf5c8d45b8aea2855d24b98dd93d0bd436: Status 404 returned error can't find the container with id 86c53154c9a358d131a3fbc41da32cdf5c8d45b8aea2855d24b98dd93d0bd436
Sep 10 14:46:00.284734 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.284683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722" event={"ID":"41064590-4693-429b-9892-eefc37ce1a68","Type":"ContainerStarted","Data":"bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492"}
Sep 10 14:46:00.284944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.284797    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:46:00.284944 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.284824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722" event={"ID":"41064590-4693-429b-9892-eefc37ce1a68","Type":"ContainerStarted","Data":"86c53154c9a358d131a3fbc41da32cdf5c8d45b8aea2855d24b98dd93d0bd436"}
Sep 10 14:46:00.286500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.286470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh" event={"ID":"f2be7141-d1bb-4a32-9218-f36497842ad5","Type":"ContainerStarted","Data":"4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12"}
Sep 10 14:46:00.286619 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.286531    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:46:00.286619 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.286555    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh" event={"ID":"f2be7141-d1bb-4a32-9218-f36497842ad5","Type":"ContainerStarted","Data":"39a12e67c2e1737930ae1480dca941cc5f3c6e63cbbfeed38c463e6b5f2d56cc"}
Sep 10 14:46:00.302316 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.302245    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722" podStartSLOduration=1.302229712 podStartE2EDuration="1.302229712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:00.300858369 +0000 UTC m=+1089.656671294" watchObservedRunningTime="2026-09-10 14:46:00.302229712 +0000 UTC m=+1089.658042657"
Sep 10 14:46:00.318182 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:00.318131    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh" podStartSLOduration=1.318113108 podStartE2EDuration="1.318113108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:00.315988042 +0000 UTC m=+1089.671800971" watchObservedRunningTime="2026-09-10 14:46:00.318113108 +0000 UTC m=+1089.673926015"
Sep 10 14:46:11.296106 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:11.296075    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:46:11.296568 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:11.296147    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:46:12.334160 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:12.334122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:14.060881 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:14.060846    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:19.033485 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:19.033453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:19.510169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:19.510117    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:22.411999 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:22.411970    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:22.512371 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:22.512334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:22.753927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:22.753893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.081194 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.081113    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.099422 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.099389    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"]
Sep 10 14:46:23.099695 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.099668    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" podUID="5615fc2c-e1d3-41fb-a02e-2c88713a3900" containerName="maas-api" containerID="cri-o://8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9" gracePeriod=30
Sep 10 14:46:23.203348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.203315    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.319688 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.319621    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.377695 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.377670    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:46:23.402031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.402000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.426478 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.426442    2560 generic.go:356] "Generic (PLEG): container finished" podID="5615fc2c-e1d3-41fb-a02e-2c88713a3900" containerID="8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9" exitCode=0
Sep 10 14:46:23.426854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.426488    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"
Sep 10 14:46:23.426854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.426525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" event={"ID":"5615fc2c-e1d3-41fb-a02e-2c88713a3900","Type":"ContainerDied","Data":"8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9"}
Sep 10 14:46:23.426854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.426581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk" event={"ID":"5615fc2c-e1d3-41fb-a02e-2c88713a3900","Type":"ContainerDied","Data":"e421243ab126ae14185c366f8f461b63d117316ad55a61e7c284429b2f251b72"}
Sep 10 14:46:23.426854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.426611    2560 scope.go:117] "RemoveContainer" containerID="8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9"
Sep 10 14:46:23.436235 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.436216    2560 scope.go:117] "RemoveContainer" containerID="8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9"
Sep 10 14:46:23.436619 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:46:23.436595    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9\": container with ID starting with 8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9 not found: ID does not exist" containerID="8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9"
Sep 10 14:46:23.436747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.436630    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9"} err="failed to get container status \"8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9\": rpc error: code = NotFound desc = could not find container \"8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9\": container with ID starting with 8f9c3fe81d6a18123010562345f743e7cfb7be7cc755f7c37c70e4bc4fb600f9 not found: ID does not exist"
Sep 10 14:46:23.467772 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.467742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.481740 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.481714    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls\") pod \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") "
Sep 10 14:46:23.481866 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.481760    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-metrics-certs\") pod \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") "
Sep 10 14:46:23.481866 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.481842    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxkcv\" (UniqueName: \"kubernetes.io/projected/5615fc2c-e1d3-41fb-a02e-2c88713a3900-kube-api-access-xxkcv\") pod \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") "
Sep 10 14:46:23.481980 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.481867    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5615fc2c-e1d3-41fb-a02e-2c88713a3900-openshift-service-ca\") pod \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\" (UID: \"5615fc2c-e1d3-41fb-a02e-2c88713a3900\") "
Sep 10 14:46:23.482258 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.482230    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5615fc2c-e1d3-41fb-a02e-2c88713a3900-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5615fc2c-e1d3-41fb-a02e-2c88713a3900" (UID: "5615fc2c-e1d3-41fb-a02e-2c88713a3900"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:46:23.484033 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.484000    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5615fc2c-e1d3-41fb-a02e-2c88713a3900" (UID: "5615fc2c-e1d3-41fb-a02e-2c88713a3900"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:46:23.484033 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.484018    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5615fc2c-e1d3-41fb-a02e-2c88713a3900-kube-api-access-xxkcv" (OuterVolumeSpecName: "kube-api-access-xxkcv") pod "5615fc2c-e1d3-41fb-a02e-2c88713a3900" (UID: "5615fc2c-e1d3-41fb-a02e-2c88713a3900"). InnerVolumeSpecName "kube-api-access-xxkcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:23.484210 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.484062    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5615fc2c-e1d3-41fb-a02e-2c88713a3900" (UID: "5615fc2c-e1d3-41fb-a02e-2c88713a3900"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:46:23.529558 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.529527    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:23.582862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.582828    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:23.582862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.582855    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5615fc2c-e1d3-41fb-a02e-2c88713a3900-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:23.582862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.582864    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxkcv\" (UniqueName: \"kubernetes.io/projected/5615fc2c-e1d3-41fb-a02e-2c88713a3900-kube-api-access-xxkcv\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:23.583120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.582874    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5615fc2c-e1d3-41fb-a02e-2c88713a3900-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:23.748797 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.748765    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"]
Sep 10 14:46:23.752233 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.752203    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5121f4-5f88d8f4d8-dqqpk"]
Sep 10 14:46:23.847479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:23.847447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:25.320347 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:25.320302    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5615fc2c-e1d3-41fb-a02e-2c88713a3900" path="/var/lib/kubelet/pods/5615fc2c-e1d3-41fb-a02e-2c88713a3900/volumes"
Sep 10 14:46:26.024542 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:26.024510    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:26.647321 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:26.647257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:26.740179 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:26.740152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:27.504090 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.504057    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:46:27.950613 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.950581    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"]
Sep 10 14:46:27.951063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.951008    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5615fc2c-e1d3-41fb-a02e-2c88713a3900" containerName="maas-api"
Sep 10 14:46:27.951063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.951019    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5615fc2c-e1d3-41fb-a02e-2c88713a3900" containerName="maas-api"
Sep 10 14:46:27.951188 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.951092    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5615fc2c-e1d3-41fb-a02e-2c88713a3900" containerName="maas-api"
Sep 10 14:46:27.955609 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.955584    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:27.959494 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.959471    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-ecfa25-serving-cert\""
Sep 10 14:46:27.971149 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:27.971116    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"]
Sep 10 14:46:28.005816 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.005787    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:28.025673 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.025638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.025799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.025682    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.025799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.025711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw2w2\" (UniqueName: \"kubernetes.io/projected/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-kube-api-access-rw2w2\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.025799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.025749    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.113176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.113145    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:28.126920 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.126890    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.127053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.126931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.127053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.126954    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw2w2\" (UniqueName: \"kubernetes.io/projected/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-kube-api-access-rw2w2\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.127053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.126988    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.127181 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:46:28.127099    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-serving-cert: secret "maas-api-e2e-shared-b-w2-ecfa25-serving-cert" not found
Sep 10 14:46:28.127181 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:46:28.127174    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls podName:2c76b901-4e73-4e90-8b9b-56cdb0263ba9 nodeName:}" failed. No retries permitted until 2026-09-10 14:46:28.62715361 +0000 UTC m=+1117.982966508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls") pod "maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" (UID: "2c76b901-4e73-4e90-8b9b-56cdb0263ba9") : secret "maas-api-e2e-shared-b-w2-ecfa25-serving-cert" not found
Sep 10 14:46:28.127646 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.127624    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.129421 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.129402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.135733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.135712    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw2w2\" (UniqueName: \"kubernetes.io/projected/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-kube-api-access-rw2w2\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.633453 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.633418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:28.633667 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:46:28.633589    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-serving-cert: secret "maas-api-e2e-shared-b-w2-ecfa25-serving-cert" not found
Sep 10 14:46:28.633667 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:46:28.633657    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls podName:2c76b901-4e73-4e90-8b9b-56cdb0263ba9 nodeName:}" failed. No retries permitted until 2026-09-10 14:46:29.633641646 +0000 UTC m=+1118.989454535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls") pod "maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" (UID: "2c76b901-4e73-4e90-8b9b-56cdb0263ba9") : secret "maas-api-e2e-shared-b-w2-ecfa25-serving-cert" not found
Sep 10 14:46:28.639590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.639560    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:28.737928 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.737893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:28.829710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:28.829678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:29.644701 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:29.644667    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:29.647243 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:29.647218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:29.767725 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:29.767694    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:29.974767 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:29.974706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:30.052375 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:30.052347    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"]
Sep 10 14:46:30.052967 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:46:30.052942    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c76b901_4e73_4e90_8b9b_56cdb0263ba9.slice/crio-6afe9d21896a1b89825d91f3483069cfdcbe6ea995765f75b3385b63385f893d WatchSource:0}: Error finding container 6afe9d21896a1b89825d91f3483069cfdcbe6ea995765f75b3385b63385f893d: Status 404 returned error can't find the container with id 6afe9d21896a1b89825d91f3483069cfdcbe6ea995765f75b3385b63385f893d
Sep 10 14:46:30.054401 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:30.054376    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:46:30.074232 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:30.074199    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:30.242717 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:30.242641    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:30.474390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:30.474356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" event={"ID":"2c76b901-4e73-4e90-8b9b-56cdb0263ba9","Type":"ContainerStarted","Data":"6afe9d21896a1b89825d91f3483069cfdcbe6ea995765f75b3385b63385f893d"}
Sep 10 14:46:30.702194 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:30.702162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:31.485679 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:31.485642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" event={"ID":"2c76b901-4e73-4e90-8b9b-56cdb0263ba9","Type":"ContainerStarted","Data":"be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566"}
Sep 10 14:46:31.485850 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:31.485823    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:31.501941 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:31.501894    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" podStartSLOduration=4.047928729 podStartE2EDuration="4.501875824s" totalImagesPullingTime="453.947095ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:31.501335937 +0000 UTC m=+1120.857148863" watchObservedRunningTime="2026-09-10 14:46:31.501875824 +0000 UTC m=+1120.857688732"
Sep 10 14:46:37.498672 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:37.498644    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:46:41.239245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:41.239213    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:41.308566 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:41.308526    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:41.414046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:41.414010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:41.510595 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:41.510490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:41.827924 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:41.827844    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:46.809224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:46.809186    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"]
Sep 10 14:46:46.904906 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:46.904872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:50.187933 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:50.187901    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:51.667179 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:51.667130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:55.380928 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:55.380892    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:46:56.960727 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:46:56.960695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:01.392056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:01.392023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:01.726317 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:01.726266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:01.829102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:01.829038    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr" podUID="8bccdbc5-8409-4ea5-8ff9-48743297a5b9" containerName="main" containerID="cri-o://e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81" gracePeriod=45
Sep 10 14:47:01.869507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:01.869480    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:02.087910 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.087886    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:47:02.167262 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167232    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tmp-dir\") pod \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") "
Sep 10 14:47:02.167456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167271    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbf9l\" (UniqueName: \"kubernetes.io/projected/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-kube-api-access-nbf9l\") pod \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") "
Sep 10 14:47:02.167456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167314    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-dshm\") pod \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") "
Sep 10 14:47:02.167456 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167422    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-home\") pod \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") "
Sep 10 14:47:02.167657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167475    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tls-certs\") pod \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") "
Sep 10 14:47:02.167657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167512    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-model-cache\") pod \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\" (UID: \"8bccdbc5-8409-4ea5-8ff9-48743297a5b9\") "
Sep 10 14:47:02.167657 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167557    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8bccdbc5-8409-4ea5-8ff9-48743297a5b9" (UID: "8bccdbc5-8409-4ea5-8ff9-48743297a5b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:47:02.167826 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167695    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-home" (OuterVolumeSpecName: "home") pod "8bccdbc5-8409-4ea5-8ff9-48743297a5b9" (UID: "8bccdbc5-8409-4ea5-8ff9-48743297a5b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:47:02.167883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167831    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:02.167883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167847    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:02.167986 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.167930    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "8bccdbc5-8409-4ea5-8ff9-48743297a5b9" (UID: "8bccdbc5-8409-4ea5-8ff9-48743297a5b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:47:02.169722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.169694    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bccdbc5-8409-4ea5-8ff9-48743297a5b9" (UID: "8bccdbc5-8409-4ea5-8ff9-48743297a5b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:47:02.169722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.169705    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-dshm" (OuterVolumeSpecName: "dshm") pod "8bccdbc5-8409-4ea5-8ff9-48743297a5b9" (UID: "8bccdbc5-8409-4ea5-8ff9-48743297a5b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:47:02.169722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.169696    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-kube-api-access-nbf9l" (OuterVolumeSpecName: "kube-api-access-nbf9l") pod "8bccdbc5-8409-4ea5-8ff9-48743297a5b9" (UID: "8bccdbc5-8409-4ea5-8ff9-48743297a5b9"). InnerVolumeSpecName "kube-api-access-nbf9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:47:02.268412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.268329    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbf9l\" (UniqueName: \"kubernetes.io/projected/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-kube-api-access-nbf9l\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:02.268412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.268359    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:02.268412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.268370    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:02.268412 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.268381    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bccdbc5-8409-4ea5-8ff9-48743297a5b9-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:02.667513 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.667422    2560 generic.go:356] "Generic (PLEG): container finished" podID="8bccdbc5-8409-4ea5-8ff9-48743297a5b9" containerID="e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81" exitCode=0
Sep 10 14:47:02.667513 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.667468    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr" event={"ID":"8bccdbc5-8409-4ea5-8ff9-48743297a5b9","Type":"ContainerDied","Data":"e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81"}
Sep 10 14:47:02.667513 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.667503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr" event={"ID":"8bccdbc5-8409-4ea5-8ff9-48743297a5b9","Type":"ContainerDied","Data":"fa797ffb7e813a8b038c76444a567bd13c8c4f1dfcf677760f5984a83a81a3ba"}
Sep 10 14:47:02.668058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.667525    2560 scope.go:117] "RemoveContainer" containerID="e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81"
Sep 10 14:47:02.668058 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.667471    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"
Sep 10 14:47:02.677611 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.677594    2560 scope.go:117] "RemoveContainer" containerID="e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81"
Sep 10 14:47:02.677887 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:47:02.677869    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81\": container with ID starting with e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81 not found: ID does not exist" containerID="e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81"
Sep 10 14:47:02.677946 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.677894    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81"} err="failed to get container status \"e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81\": rpc error: code = NotFound desc = could not find container \"e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81\": container with ID starting with e0c7e6243f926ef54355f59d3ece8598d7b02283162f8b2edd1c3d69147b1a81 not found: ID does not exist"
Sep 10 14:47:02.691349 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.691322    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"]
Sep 10 14:47:02.695720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:02.695698    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2dccd5/test-model-2dccd5-kserve-6bf74f4787-2fmpr"]
Sep 10 14:47:03.319475 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:03.319440    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bccdbc5-8409-4ea5-8ff9-48743297a5b9" path="/var/lib/kubelet/pods/8bccdbc5-8409-4ea5-8ff9-48743297a5b9/volumes"
Sep 10 14:47:04.964686 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:04.964651    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:05.062348 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:05.062319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:06.137022 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:06.136992    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:06.199616 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:06.199577    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:06.299725 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:06.299674    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:06.602559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:06.602524    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:06.694013 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:06.693974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:11.013177 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.013139    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"]
Sep 10 14:47:11.013664 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.013646    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bccdbc5-8409-4ea5-8ff9-48743297a5b9" containerName="main"
Sep 10 14:47:11.013708 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.013665    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bccdbc5-8409-4ea5-8ff9-48743297a5b9" containerName="main"
Sep 10 14:47:11.013749 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.013728    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bccdbc5-8409-4ea5-8ff9-48743297a5b9" containerName="main"
Sep 10 14:47:11.018146 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.018129    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.020443 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.020423    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9b1e67\"/\"distinct-2-w0-9b1e67-kserve-self-signed-certs\""
Sep 10 14:47:11.026888 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.026867    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"]
Sep 10 14:47:11.157863 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.157834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-tmp-dir\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.158046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.157906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-model-cache\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.158046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.157934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-dshm\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.158046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.157952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bbeb6-0249-433c-a2cd-7bcfb3823259-tls-certs\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.158046 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.158027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zrwp\" (UniqueName: \"kubernetes.io/projected/585bbeb6-0249-433c-a2cd-7bcfb3823259-kube-api-access-4zrwp\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.158239 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.158091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-home\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259379 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-tmp-dir\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-model-cache\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259628 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-dshm\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bbeb6-0249-433c-a2cd-7bcfb3823259-tls-certs\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zrwp\" (UniqueName: \"kubernetes.io/projected/585bbeb6-0249-433c-a2cd-7bcfb3823259-kube-api-access-4zrwp\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-home\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259735    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-model-cache\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.259895 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-tmp-dir\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.260022 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.259995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-home\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.261791 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.261773    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-dshm\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.262170 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.262151    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bbeb6-0249-433c-a2cd-7bcfb3823259-tls-certs\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.267256 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.267206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zrwp\" (UniqueName: \"kubernetes.io/projected/585bbeb6-0249-433c-a2cd-7bcfb3823259-kube-api-access-4zrwp\") pod \"distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") " pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.328555 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.328519    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:11.534860 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.534755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:11.601158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.601125    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"]
Sep 10 14:47:11.602862 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:47:11.602835    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod585bbeb6_0249_433c_a2cd_7bcfb3823259.slice/crio-b037e045adb78339230b740507748a5e9977a98b53d219e9d973690a49ffae80 WatchSource:0}: Error finding container b037e045adb78339230b740507748a5e9977a98b53d219e9d973690a49ffae80: Status 404 returned error can't find the container with id b037e045adb78339230b740507748a5e9977a98b53d219e9d973690a49ffae80
Sep 10 14:47:11.723746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.723708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc" event={"ID":"585bbeb6-0249-433c-a2cd-7bcfb3823259","Type":"ContainerStarted","Data":"30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c"}
Sep 10 14:47:11.723880 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.723760    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc" event={"ID":"585bbeb6-0249-433c-a2cd-7bcfb3823259","Type":"ContainerStarted","Data":"b037e045adb78339230b740507748a5e9977a98b53d219e9d973690a49ffae80"}
Sep 10 14:47:11.740537 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:11.740488    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc" podStartSLOduration=1.740467741 podStartE2EDuration="1.740467741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:11.738102964 +0000 UTC m=+1161.093915871" watchObservedRunningTime="2026-09-10 14:47:11.740467741 +0000 UTC m=+1161.096280652"
Sep 10 14:47:16.427689 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:16.427598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:21.328983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:21.328948    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:21.328983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:21.328989    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:21.341497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:21.341465    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:21.791450 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:21.791421    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:47:23.562857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:23.562821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:23.833427 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:23.833347    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:24.381163 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:24.381128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:24.580951 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:24.580917    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:47:24.685054 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:24.684973    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:24.784569 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:24.784538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:28.112977 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:28.112944    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:28.222617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:28.222584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.243384 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.243352    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"]
Sep 10 14:47:29.247622 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.247598    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.249859 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.249836    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-12b6f6-serving-cert\""
Sep 10 14:47:29.256434 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.256410    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"]
Sep 10 14:47:29.333015 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.332972    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bd9805-96e2-4700-a5f2-f6ced2d20048-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.333015 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.333010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lcnj\" (UniqueName: \"kubernetes.io/projected/c4bd9805-96e2-4700-a5f2-f6ced2d20048-kube-api-access-5lcnj\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.333225 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.333032    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.333225 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.333154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.406750 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.406717    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.434037 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.434008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.434156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.434084    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bd9805-96e2-4700-a5f2-f6ced2d20048-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.434156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.434102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lcnj\" (UniqueName: \"kubernetes.io/projected/c4bd9805-96e2-4700-a5f2-f6ced2d20048-kube-api-access-5lcnj\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.434156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.434123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.434156 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:47:29.434145    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-serving-cert: secret "maas-api-e2e-shared-a-w3-12b6f6-serving-cert" not found
Sep 10 14:47:29.434324 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:47:29.434244    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls podName:c4bd9805-96e2-4700-a5f2-f6ced2d20048 nodeName:}" failed. No retries permitted until 2026-09-10 14:47:29.93422702 +0000 UTC m=+1179.290039922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls") pod "maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" (UID: "c4bd9805-96e2-4700-a5f2-f6ced2d20048") : secret "maas-api-e2e-shared-a-w3-12b6f6-serving-cert" not found
Sep 10 14:47:29.434847 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.434824    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bd9805-96e2-4700-a5f2-f6ced2d20048-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.436667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.436640    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.444505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.444482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lcnj\" (UniqueName: \"kubernetes.io/projected/c4bd9805-96e2-4700-a5f2-f6ced2d20048-kube-api-access-5lcnj\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.483693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.483650    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.536614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.536531    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.677178 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.677141    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.777582 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.777544    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.877574 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.877499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.940093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.940062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.942439 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.942406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:29.980357 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.980320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:29.994471 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.994446    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"]
Sep 10 14:47:29.998289 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:29.998262    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.000445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.000419    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-12b6f6\""
Sep 10 14:47:30.000445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.000446    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-12b6f6-dockercfg-cr222\""
Sep 10 14:47:30.008336 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.008314    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"]
Sep 10 14:47:30.041198 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.041168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svm4j\" (UniqueName: \"kubernetes.io/projected/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-kube-api-access-svm4j\") pod \"payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.041355 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.041299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.045184 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.045163    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:30.142862 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.142777    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.143012 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.142863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svm4j\" (UniqueName: \"kubernetes.io/projected/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-kube-api-access-svm4j\") pod \"payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.143505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.143485    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.150035 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.150012    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svm4j\" (UniqueName: \"kubernetes.io/projected/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-kube-api-access-svm4j\") pod \"payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.163882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.163856    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:30.311544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.311510    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.441580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.441547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"]
Sep 10 14:47:30.442637 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:47:30.442601    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bd9805_96e2_4700_a5f2_f6ced2d20048.slice/crio-7ac9df9b96321a3f77fd50aca15b46203d35c35106aad2b9238d166dbdb8f7b5 WatchSource:0}: Error finding container 7ac9df9b96321a3f77fd50aca15b46203d35c35106aad2b9238d166dbdb8f7b5: Status 404 returned error can't find the container with id 7ac9df9b96321a3f77fd50aca15b46203d35c35106aad2b9238d166dbdb8f7b5
Sep 10 14:47:30.587289 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.587250    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"]
Sep 10 14:47:30.588094 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:47:30.588064    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5334ba06_f4c5_4c5c_8e07_ecfedaa33039.slice/crio-a96d5bcf1153abcad75284545df85c7e850be6e19a365f1c8c9d038ec2819e44 WatchSource:0}: Error finding container a96d5bcf1153abcad75284545df85c7e850be6e19a365f1c8c9d038ec2819e44: Status 404 returned error can't find the container with id a96d5bcf1153abcad75284545df85c7e850be6e19a365f1c8c9d038ec2819e44
Sep 10 14:47:30.836675 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.836594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" event={"ID":"c4bd9805-96e2-4700-a5f2-f6ced2d20048","Type":"ContainerStarted","Data":"7ac9df9b96321a3f77fd50aca15b46203d35c35106aad2b9238d166dbdb8f7b5"}
Sep 10 14:47:30.838699 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.838672    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4" event={"ID":"5334ba06-f4c5-4c5c-8e07-ecfedaa33039","Type":"ContainerStarted","Data":"c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f"}
Sep 10 14:47:30.838812 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.838721    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4" event={"ID":"5334ba06-f4c5-4c5c-8e07-ecfedaa33039","Type":"ContainerStarted","Data":"a96d5bcf1153abcad75284545df85c7e850be6e19a365f1c8c9d038ec2819e44"}
Sep 10 14:47:30.838872 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.838805    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:30.855747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:30.855703    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4" podStartSLOduration=1.8556869329999999 podStartE2EDuration="1.855686933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:30.853505599 +0000 UTC m=+1180.209318526" watchObservedRunningTime="2026-09-10 14:47:30.855686933 +0000 UTC m=+1180.211499839"
Sep 10 14:47:31.846077 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:31.846042    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" event={"ID":"c4bd9805-96e2-4700-a5f2-f6ced2d20048","Type":"ContainerStarted","Data":"391ec1932ac93b6cfbe123d574454831f329ce570aa6b50836e841e07d711c81"}
Sep 10 14:47:31.846491 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:31.846403    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:31.862794 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:31.862749    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" podStartSLOduration=2.496296539 podStartE2EDuration="2.862734058s" totalImagesPullingTime="366.437519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:31.859579313 +0000 UTC m=+1181.215392231" watchObservedRunningTime="2026-09-10 14:47:31.862734058 +0000 UTC m=+1181.218546965"
Sep 10 14:47:34.883413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:34.883373    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:36.584671 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:36.584632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:37.857667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:37.857640    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 14:47:41.848791 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:41.848756    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 14:47:47.226490 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:47.226453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:51.629799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:51.629767    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:53.097591 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:53.097558    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:53.440954 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:53.440924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:53.736560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:53.736476    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:54.245022 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:54.244988    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:57.300864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:57.300830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:57.400562 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:57.400528    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:58.604793 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:58.604758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:58.704940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:58.704911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:58.792961 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:58.792932    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:47:59.027395 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:59.027365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:59.352129 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:59.352044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:59.496133 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:47:59.496100    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:09.620169 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.620139    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"]
Sep 10 14:48:09.631848 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.631805    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.634156 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.634105    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"]
Sep 10 14:48:09.636024 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.635982    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e43b34d1\"/\"kube-root-ca.crt\""
Sep 10 14:48:09.636455 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.636015    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e43b34d1\"/\"openshift-service-ca.crt\""
Sep 10 14:48:09.636883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.636077    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e43b34d1\"/\"default-dockercfg-dcvln\""
Sep 10 14:48:09.636883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.636095    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e43b34d1\"/\"e2e-lifecycle-model-e43b34d1-kserve-self-signed-certs\""
Sep 10 14:48:09.648130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.648104    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:09.727812 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.727772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-tmp-dir\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.727812 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.727812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-model-cache\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.728028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.727832    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d609abee-9458-491c-9c6e-eab1c42dedde-tls-certs\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.728028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.727929    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-home\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.728028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.727988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95qg\" (UniqueName: \"kubernetes.io/projected/d609abee-9458-491c-9c6e-eab1c42dedde-kube-api-access-l95qg\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.728134 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.728032    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-dshm\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829042 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95qg\" (UniqueName: \"kubernetes.io/projected/d609abee-9458-491c-9c6e-eab1c42dedde-kube-api-access-l95qg\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829042 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-dshm\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829313 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829103    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-tmp-dir\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829313 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-model-cache\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829313 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d609abee-9458-491c-9c6e-eab1c42dedde-tls-certs\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-home\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829594 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829570    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-tmp-dir\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-model-cache\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.829710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.829640    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-home\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.831513 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.831490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-dshm\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.831689 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.831671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d609abee-9458-491c-9c6e-eab1c42dedde-tls-certs\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.837271 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.837249    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95qg\" (UniqueName: \"kubernetes.io/projected/d609abee-9458-491c-9c6e-eab1c42dedde-kube-api-access-l95qg\") pod \"e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") " pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:09.950158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:09.950125    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:10.222328 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:10.222298    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"]
Sep 10 14:48:10.223497 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:48:10.223471    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd609abee_9458_491c_9c6e_eab1c42dedde.slice/crio-e9185ee8313a58cb806e2c3eb22763901771b78ca27a07c01ae1d49d1c1a67b7 WatchSource:0}: Error finding container e9185ee8313a58cb806e2c3eb22763901771b78ca27a07c01ae1d49d1c1a67b7: Status 404 returned error can't find the container with id e9185ee8313a58cb806e2c3eb22763901771b78ca27a07c01ae1d49d1c1a67b7
Sep 10 14:48:11.093313 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:11.093258    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8" event={"ID":"d609abee-9458-491c-9c6e-eab1c42dedde","Type":"ContainerStarted","Data":"45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52"}
Sep 10 14:48:11.093758 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:11.093317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8" event={"ID":"d609abee-9458-491c-9c6e-eab1c42dedde","Type":"ContainerStarted","Data":"e9185ee8313a58cb806e2c3eb22763901771b78ca27a07c01ae1d49d1c1a67b7"}
Sep 10 14:48:11.113535 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:11.113489    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8" podStartSLOduration=2.11347437 podStartE2EDuration="2.11347437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:11.110300423 +0000 UTC m=+1220.466113345" watchObservedRunningTime="2026-09-10 14:48:11.11347437 +0000 UTC m=+1220.469287277"
Sep 10 14:48:11.912439 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:11.912404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:17.848378 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:17.848339    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:19.950611 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:19.950569    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:19.951071 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:19.950628    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:19.963402 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:19.963372    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:20.163029 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:20.162992    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:48:22.315369 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:22.315272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:44.186447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:44.186370    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:48:45.834233 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.834193    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"]
Sep 10 14:48:45.852498 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.852470    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:45.852771 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.852741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"]
Sep 10 14:48:45.854610 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.854584    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9b1e67\"/\"unconfigured-w0-9b1e67-kserve-self-signed-certs\""
Sep 10 14:48:45.870864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.870830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:45.977557 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.977528    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb9602d-68d3-40c2-8561-805c2014e908-tls-certs\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:45.977715 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.977609    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-dshm\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:45.977715 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.977653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-model-cache\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:45.977715 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.977677    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv2wr\" (UniqueName: \"kubernetes.io/projected/9bb9602d-68d3-40c2-8561-805c2014e908-kube-api-access-qv2wr\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:45.977825 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.977730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-tmp-dir\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:45.977825 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:45.977750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-home\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.078729 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.078693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-tmp-dir\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.078882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.078746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-home\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.078882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.078856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb9602d-68d3-40c2-8561-805c2014e908-tls-certs\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.078980 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.078891    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-dshm\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.078980 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.078923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-model-cache\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.078980 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.078967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2wr\" (UniqueName: \"kubernetes.io/projected/9bb9602d-68d3-40c2-8561-805c2014e908-kube-api-access-qv2wr\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.079181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.079137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-tmp-dir\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.079250 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.079181    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-home\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.079377 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.079349    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-model-cache\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.081111 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.081091    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-dshm\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.081385 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.081369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb9602d-68d3-40c2-8561-805c2014e908-tls-certs\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.086523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.086464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv2wr\" (UniqueName: \"kubernetes.io/projected/9bb9602d-68d3-40c2-8561-805c2014e908-kube-api-access-qv2wr\") pod \"unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") " pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.164474 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.164422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:46.438683 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:46.438655    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"]
Sep 10 14:48:46.439990 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:48:46.439962    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bb9602d_68d3_40c2_8561_805c2014e908.slice/crio-e665b54800833f6e9b0303b443c537ab4fd5ccb2a30d8824e4f0cdd7dbef51be WatchSource:0}: Error finding container e665b54800833f6e9b0303b443c537ab4fd5ccb2a30d8824e4f0cdd7dbef51be: Status 404 returned error can't find the container with id e665b54800833f6e9b0303b443c537ab4fd5ccb2a30d8824e4f0cdd7dbef51be
Sep 10 14:48:47.322505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:47.322467    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2" event={"ID":"9bb9602d-68d3-40c2-8561-805c2014e908","Type":"ContainerStarted","Data":"eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f"}
Sep 10 14:48:47.323147 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:47.322515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2" event={"ID":"9bb9602d-68d3-40c2-8561-805c2014e908","Type":"ContainerStarted","Data":"e665b54800833f6e9b0303b443c537ab4fd5ccb2a30d8824e4f0cdd7dbef51be"}
Sep 10 14:48:47.340989 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:47.340937    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2" podStartSLOduration=2.340922247 podStartE2EDuration="2.340922247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:47.338686422 +0000 UTC m=+1256.694499331" watchObservedRunningTime="2026-09-10 14:48:47.340922247 +0000 UTC m=+1256.696735152"
Sep 10 14:48:56.165072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:56.165036    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:56.165585 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:56.165343    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:56.177826 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:56.177800    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:48:56.389314 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:48:56.389274    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:49:15.378430 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:15.378397    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:49:17.071684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:17.071648    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:23.066082 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:23.066045    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:26.344173 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:26.344138    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:26.398413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:26.398378    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:30.798845 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:30.798797    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:38.511751 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:38.511711    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:38.613528 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:38.613491    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:38.714741 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:38.714715    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:38.782181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:38.782079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:41.977579 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:41.977536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:46.962948 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:46.962903    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"]
Sep 10 14:49:47.058142 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:47.058112    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:51.878226 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:51.878190    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:49:53.460992 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:49:53.460955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:01.983919 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:01.983874    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8" podUID="d609abee-9458-491c-9c6e-eab1c42dedde" containerName="main" containerID="cri-o://45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52" gracePeriod=45
Sep 10 14:50:02.222643 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.222614    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:50:02.340810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.340707    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l95qg\" (UniqueName: \"kubernetes.io/projected/d609abee-9458-491c-9c6e-eab1c42dedde-kube-api-access-l95qg\") pod \"d609abee-9458-491c-9c6e-eab1c42dedde\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") "
Sep 10 14:50:02.340810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.340754    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d609abee-9458-491c-9c6e-eab1c42dedde-tls-certs\") pod \"d609abee-9458-491c-9c6e-eab1c42dedde\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") "
Sep 10 14:50:02.340810 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.340798    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-home\") pod \"d609abee-9458-491c-9c6e-eab1c42dedde\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") "
Sep 10 14:50:02.341101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.340819    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-dshm\") pod \"d609abee-9458-491c-9c6e-eab1c42dedde\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") "
Sep 10 14:50:02.341101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.340859    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-model-cache\") pod \"d609abee-9458-491c-9c6e-eab1c42dedde\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") "
Sep 10 14:50:02.341101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.340920    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-tmp-dir\") pod \"d609abee-9458-491c-9c6e-eab1c42dedde\" (UID: \"d609abee-9458-491c-9c6e-eab1c42dedde\") "
Sep 10 14:50:02.341101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.341084    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-home" (OuterVolumeSpecName: "home") pod "d609abee-9458-491c-9c6e-eab1c42dedde" (UID: "d609abee-9458-491c-9c6e-eab1c42dedde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:50:02.341350 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.341167    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-model-cache" (OuterVolumeSpecName: "model-cache") pod "d609abee-9458-491c-9c6e-eab1c42dedde" (UID: "d609abee-9458-491c-9c6e-eab1c42dedde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:50:02.341350 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.341260    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d609abee-9458-491c-9c6e-eab1c42dedde" (UID: "d609abee-9458-491c-9c6e-eab1c42dedde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:50:02.341453 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.341370    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:02.341453 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.341384    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:02.341453 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.341393    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:02.343055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.343030    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-dshm" (OuterVolumeSpecName: "dshm") pod "d609abee-9458-491c-9c6e-eab1c42dedde" (UID: "d609abee-9458-491c-9c6e-eab1c42dedde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:50:02.343185 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.343061    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d609abee-9458-491c-9c6e-eab1c42dedde-kube-api-access-l95qg" (OuterVolumeSpecName: "kube-api-access-l95qg") pod "d609abee-9458-491c-9c6e-eab1c42dedde" (UID: "d609abee-9458-491c-9c6e-eab1c42dedde"). InnerVolumeSpecName "kube-api-access-l95qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:02.343185 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.343091    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d609abee-9458-491c-9c6e-eab1c42dedde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d609abee-9458-491c-9c6e-eab1c42dedde" (UID: "d609abee-9458-491c-9c6e-eab1c42dedde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:50:02.442599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.442551    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l95qg\" (UniqueName: \"kubernetes.io/projected/d609abee-9458-491c-9c6e-eab1c42dedde-kube-api-access-l95qg\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:02.442599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.442591    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d609abee-9458-491c-9c6e-eab1c42dedde-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:02.442599 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.442603    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d609abee-9458-491c-9c6e-eab1c42dedde-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:02.775771 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.775735    2560 generic.go:356] "Generic (PLEG): container finished" podID="d609abee-9458-491c-9c6e-eab1c42dedde" containerID="45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52" exitCode=0
Sep 10 14:50:02.775771 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.775771    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"
Sep 10 14:50:02.776017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.775830    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8" event={"ID":"d609abee-9458-491c-9c6e-eab1c42dedde","Type":"ContainerDied","Data":"45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52"}
Sep 10 14:50:02.776017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.775871    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8" event={"ID":"d609abee-9458-491c-9c6e-eab1c42dedde","Type":"ContainerDied","Data":"e9185ee8313a58cb806e2c3eb22763901771b78ca27a07c01ae1d49d1c1a67b7"}
Sep 10 14:50:02.776017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.775889    2560 scope.go:117] "RemoveContainer" containerID="45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52"
Sep 10 14:50:02.786529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.786510    2560 scope.go:117] "RemoveContainer" containerID="45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52"
Sep 10 14:50:02.786869 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:50:02.786849    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52\": container with ID starting with 45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52 not found: ID does not exist" containerID="45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52"
Sep 10 14:50:02.786973 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.786877    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52"} err="failed to get container status \"45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52\": rpc error: code = NotFound desc = could not find container \"45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52\": container with ID starting with 45b8cbad35605127007cfc12bb2dc77c0e74f6e375e5f582608afdb03806ee52 not found: ID does not exist"
Sep 10 14:50:02.797586 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.797544    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"]
Sep 10 14:50:02.800895 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:02.800870    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e43b34d1/e2e-lifecycle-model-e43b34d1-kserve-76f4d75595-j68r8"]
Sep 10 14:50:03.319053 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:03.319020    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d609abee-9458-491c-9c6e-eab1c42dedde" path="/var/lib/kubelet/pods/d609abee-9458-491c-9c6e-eab1c42dedde/volumes"
Sep 10 14:50:13.467255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:13.467161    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:22.996401 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:22.996369    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:23.096321 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:23.096261    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:23.195909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:23.195875    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:23.900710 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:23.900654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:24.981808 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:24.981768    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:50:29.473016 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:29.472978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:35.149056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:35.149007    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:35.217841 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:35.217812    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:35.415269 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:35.415170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:36.692330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:36.692293    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:36.817661 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:36.817632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:36.918479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:36.918440    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:37.542221 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:37.542185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:37.641093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:37.641062    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:37.744413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:37.744357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:39.024137 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:39.024100    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:44.685924 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:44.685872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:44.785879 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:44.785850    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:45.354550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:45.354518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:50.262934 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:50.262903    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:52.497690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:52.497653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:52.616364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:52.616317    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:52.745654 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:52.745616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:54.060609 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:50:54.060557    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:03.223746 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:03.223704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:03.447865 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:03.447834    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:03.977914 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:03.977880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:18.983830 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:18.983793    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:51:20.304834 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:20.304785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:24.355500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:24.355465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:30.162939 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:30.162906    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:30.263262 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:30.263231    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:30.525610 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:30.525574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:30.833661 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:30.833560    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:31.229971 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:31.229933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:33.778761 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:33.778728    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:33.910423 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:33.910394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:35.062517 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.062483    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:35.121517 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.121487    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:35.226579 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.226536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:35.529092 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.529057    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:35.624813 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.624763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:35.698728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.698695    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"]
Sep 10 14:51:35.699186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.699152    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d609abee-9458-491c-9c6e-eab1c42dedde" containerName="main"
Sep 10 14:51:35.699186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.699186    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d609abee-9458-491c-9c6e-eab1c42dedde" containerName="main"
Sep 10 14:51:35.699319 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.699296    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d609abee-9458-491c-9c6e-eab1c42dedde" containerName="main"
Sep 10 14:51:35.702636 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.702622    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:35.704827 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.704802    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-6eb765\""
Sep 10 14:51:35.704933 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.704905    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-6eb765-dockercfg-nsrjc\""
Sep 10 14:51:35.712864 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.712840    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"]
Sep 10 14:51:35.855138 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.855044    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nngkm\" (UniqueName: \"kubernetes.io/projected/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-kube-api-access-nngkm\") pod \"payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:35.855138 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.855108    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:35.956709 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.956666    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nngkm\" (UniqueName: \"kubernetes.io/projected/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-kube-api-access-nngkm\") pod \"payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:35.956941 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.956760    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:35.957647 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.957616    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:35.964824 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:35.964794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nngkm\" (UniqueName: \"kubernetes.io/projected/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-kube-api-access-nngkm\") pod \"payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:36.014205 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:36.014168    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:36.290807 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:36.290769    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"]
Sep 10 14:51:36.291697 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:51:36.291670    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1f670e_3d8f_415c_b1ce_51e48eb96c8b.slice/crio-f92916f318724a016647c3da49e97c71818c928a2e9ac7bd7316d24636394951 WatchSource:0}: Error finding container f92916f318724a016647c3da49e97c71818c928a2e9ac7bd7316d24636394951: Status 404 returned error can't find the container with id f92916f318724a016647c3da49e97c71818c928a2e9ac7bd7316d24636394951
Sep 10 14:51:36.361338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:36.361305    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z" event={"ID":"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b","Type":"ContainerStarted","Data":"f92916f318724a016647c3da49e97c71818c928a2e9ac7bd7316d24636394951"}
Sep 10 14:51:37.368509 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:37.368473    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z" event={"ID":"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b","Type":"ContainerStarted","Data":"17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a"}
Sep 10 14:51:37.368882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:37.368575    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:37.385998 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:37.385942    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z" podStartSLOduration=2.385925994 podStartE2EDuration="2.385925994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:51:37.382449918 +0000 UTC m=+1426.738262836" watchObservedRunningTime="2026-09-10 14:51:37.385925994 +0000 UTC m=+1426.741738936"
Sep 10 14:51:44.703720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:44.703645    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:48.378311 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:48.378261    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:51:50.947572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:50.947538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:52.118224 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.118193    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"]
Sep 10 14:51:52.121833 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.121808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.123801 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.123780    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5b0323\"/\"e2e-conflict-a-f6cac41e-kserve-self-signed-certs\""
Sep 10 14:51:52.129731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.129708    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"]
Sep 10 14:51:52.212684 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.212642    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-dshm\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.212870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.212721    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-home\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.212870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.212743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-model-cache\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.212870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.212777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tmp-dir\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.212870 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.212816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tls-certs\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.213026 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.212921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmchc\" (UniqueName: \"kubernetes.io/projected/e90498b0-6417-4d3b-bd43-f5ed29eca51b-kube-api-access-nmchc\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314079 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmchc\" (UniqueName: \"kubernetes.io/projected/e90498b0-6417-4d3b-bd43-f5ed29eca51b-kube-api-access-nmchc\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-dshm\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-home\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314162    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-model-cache\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314183    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tmp-dir\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314251 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314198    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tls-certs\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314575 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-model-cache\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-home\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.314772 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.314671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tmp-dir\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.316714 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.316688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-dshm\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.316909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.316894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tls-certs\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.320884 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.320859    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:52.321507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.321488    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmchc\" (UniqueName: \"kubernetes.io/projected/e90498b0-6417-4d3b-bd43-f5ed29eca51b-kube-api-access-nmchc\") pod \"e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.432269 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.432187    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:51:52.705933 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:52.705907    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"]
Sep 10 14:51:52.708343 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:51:52.708315    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode90498b0_6417_4d3b_bd43_f5ed29eca51b.slice/crio-8b144a161836094b7fd9a387e95531d28c3ad7ab5f822ba9142cfe00254029e0 WatchSource:0}: Error finding container 8b144a161836094b7fd9a387e95531d28c3ad7ab5f822ba9142cfe00254029e0: Status 404 returned error can't find the container with id 8b144a161836094b7fd9a387e95531d28c3ad7ab5f822ba9142cfe00254029e0
Sep 10 14:51:53.470035 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:53.469996    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46" event={"ID":"e90498b0-6417-4d3b-bd43-f5ed29eca51b","Type":"ContainerStarted","Data":"4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763"}
Sep 10 14:51:53.470445 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:53.470047    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46" event={"ID":"e90498b0-6417-4d3b-bd43-f5ed29eca51b","Type":"ContainerStarted","Data":"8b144a161836094b7fd9a387e95531d28c3ad7ab5f822ba9142cfe00254029e0"}
Sep 10 14:51:53.487808 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:53.487758    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46" podStartSLOduration=1.487741866 podStartE2EDuration="1.487741866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:51:53.485814182 +0000 UTC m=+1442.841627104" watchObservedRunningTime="2026-09-10 14:51:53.487741866 +0000 UTC m=+1442.843554774"
Sep 10 14:51:56.961448 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:56.961406    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:58.730416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:58.730385    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:59.189134 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:59.189102    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:59.473577 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:59.473495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:59.571101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:51:59.571072    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:02.433148 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:02.433109    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:52:02.433596 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:02.433179    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:52:02.446245 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:02.446215    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:52:02.538627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:02.538597    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:52:03.030802 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:03.030764    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:03.131130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:03.131090    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:04.184692 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.184658    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:04.257958 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.257925    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:04.451918 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.451879    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:04.765991 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.765910    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:04.861029 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.860995    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:04.950476 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.950441    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"]
Sep 10 14:52:04.954407 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.954386    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:04.956694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.956662    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f1b717\""
Sep 10 14:52:04.956906 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.956887    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f1b717-dockercfg-wzzlb\""
Sep 10 14:52:04.964129 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:04.964104    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"]
Sep 10 14:52:05.048883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.048782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ecb0487-f599-4a1d-9328-93f92aa7030e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.048883 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.048859    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlf7w\" (UniqueName: \"kubernetes.io/projected/5ecb0487-f599-4a1d-9328-93f92aa7030e-kube-api-access-vlf7w\") pod \"payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.150261 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.150217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ecb0487-f599-4a1d-9328-93f92aa7030e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.150489 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.150330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlf7w\" (UniqueName: \"kubernetes.io/projected/5ecb0487-f599-4a1d-9328-93f92aa7030e-kube-api-access-vlf7w\") pod \"payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.150921 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.150902    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ecb0487-f599-4a1d-9328-93f92aa7030e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.158225 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.158198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlf7w\" (UniqueName: \"kubernetes.io/projected/5ecb0487-f599-4a1d-9328-93f92aa7030e-kube-api-access-vlf7w\") pod \"payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.268273 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.268237    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:05.545614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:05.545586    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"]
Sep 10 14:52:05.547148 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:52:05.547109    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ecb0487_f599_4a1d_9328_93f92aa7030e.slice/crio-016537d22c8b16571aec13b0bc936bd14256644dce3eaf334a923f45c7716bf0 WatchSource:0}: Error finding container 016537d22c8b16571aec13b0bc936bd14256644dce3eaf334a923f45c7716bf0: Status 404 returned error can't find the container with id 016537d22c8b16571aec13b0bc936bd14256644dce3eaf334a923f45c7716bf0
Sep 10 14:52:06.199351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:06.199311    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:52:06.554606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:06.554507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk" event={"ID":"5ecb0487-f599-4a1d-9328-93f92aa7030e","Type":"ContainerStarted","Data":"b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd"}
Sep 10 14:52:06.554606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:06.554569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk" event={"ID":"5ecb0487-f599-4a1d-9328-93f92aa7030e","Type":"ContainerStarted","Data":"016537d22c8b16571aec13b0bc936bd14256644dce3eaf334a923f45c7716bf0"}
Sep 10 14:52:06.554606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:06.554597    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:06.569168 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:06.569116    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk" podStartSLOduration=2.569100499 podStartE2EDuration="2.569100499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:06.567723463 +0000 UTC m=+1455.923536380" watchObservedRunningTime="2026-09-10 14:52:06.569100499 +0000 UTC m=+1455.924913406"
Sep 10 14:52:12.565227 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:12.565171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:17.562975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:17.562933    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:52:28.035502 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:28.035466    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:31.226478 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:31.226435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:36.332666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.332631    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"]
Sep 10 14:52:36.337468 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.337445    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.339735 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.339702    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5b0323\"/\"e2e-conflict-b-f6cac41e-kserve-self-signed-certs\""
Sep 10 14:52:36.344380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.344353    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"]
Sep 10 14:52:36.357464 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.357423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:36.361498 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.361473    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tls-certs\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.361609 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.361526    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgvtn\" (UniqueName: \"kubernetes.io/projected/7bd58eb8-75c8-462f-bf2e-58919503e4a4-kube-api-access-vgvtn\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.361673 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.361593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-dshm\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.361733 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.361662    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tmp-dir\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.362004 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.361982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-model-cache\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.362100 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.362043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-home\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-model-cache\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-home\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463340    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tls-certs\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463377    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgvtn\" (UniqueName: \"kubernetes.io/projected/7bd58eb8-75c8-462f-bf2e-58919503e4a4-kube-api-access-vgvtn\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-dshm\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463488 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tmp-dir\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463773 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-model-cache\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463773 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463742    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-home\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.463877 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.463809    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tmp-dir\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.465774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.465756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-dshm\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.465918 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.465898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tls-certs\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.471423 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.471403    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgvtn\" (UniqueName: \"kubernetes.io/projected/7bd58eb8-75c8-462f-bf2e-58919503e4a4-kube-api-access-vgvtn\") pod \"e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") " pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.648846 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.648729    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:36.924627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:36.924595    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"]
Sep 10 14:52:36.926828 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:52:36.926796    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bd58eb8_75c8_462f_bf2e_58919503e4a4.slice/crio-c38e981f1028f82308448a332b3490eafe523a336403cc6442996d58e4d4b79b WatchSource:0}: Error finding container c38e981f1028f82308448a332b3490eafe523a336403cc6442996d58e4d4b79b: Status 404 returned error can't find the container with id c38e981f1028f82308448a332b3490eafe523a336403cc6442996d58e4d4b79b
Sep 10 14:52:37.465310 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:37.465254    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:37.756713 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:37.756607    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd" event={"ID":"7bd58eb8-75c8-462f-bf2e-58919503e4a4","Type":"ContainerStarted","Data":"bd522403d9380acf74997870dd8cb76c802778344e16915e41bfaf23a3d11b1c"}
Sep 10 14:52:37.756713 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:37.756660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd" event={"ID":"7bd58eb8-75c8-462f-bf2e-58919503e4a4","Type":"ContainerStarted","Data":"c38e981f1028f82308448a332b3490eafe523a336403cc6442996d58e4d4b79b"}
Sep 10 14:52:37.773517 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:37.773456    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd" podStartSLOduration=1.77343958 podStartE2EDuration="1.77343958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:37.772050914 +0000 UTC m=+1487.127863842" watchObservedRunningTime="2026-09-10 14:52:37.77343958 +0000 UTC m=+1487.129252487"
Sep 10 14:52:41.138428 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:41.138395    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:46.649466 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:46.649425    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:46.649945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:46.649485    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:46.663413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:46.663384    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:46.794723 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:46.794689    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"]
Sep 10 14:52:46.795857 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:46.795821    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z" podUID="ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" containerName="payload-pre-processing" containerID="cri-o://17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a" gracePeriod=30
Sep 10 14:52:46.828017 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:46.827987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:52:47.051442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.051414    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:52:47.178135 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.178047    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nngkm\" (UniqueName: \"kubernetes.io/projected/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-kube-api-access-nngkm\") pod \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") "
Sep 10 14:52:47.178135 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.178092    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-plugins-config-volume\") pod \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\" (UID: \"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b\") "
Sep 10 14:52:47.178528 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.178496    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" (UID: "ec1f670e-3d8f-415c-b1ce-51e48eb96c8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:52:47.180183 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.180161    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-kube-api-access-nngkm" (OuterVolumeSpecName: "kube-api-access-nngkm") pod "ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" (UID: "ec1f670e-3d8f-415c-b1ce-51e48eb96c8b"). InnerVolumeSpecName "kube-api-access-nngkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:52:47.279297 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.279255    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nngkm\" (UniqueName: \"kubernetes.io/projected/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-kube-api-access-nngkm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:52:47.279465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.279319    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:52:47.423538 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.423506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:47.490734 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.490703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:47.821760 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.821671    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" containerID="17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a" exitCode=0
Sep 10 14:52:47.821760 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.821715    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"
Sep 10 14:52:47.821760 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.821753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z" event={"ID":"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b","Type":"ContainerDied","Data":"17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a"}
Sep 10 14:52:47.822334 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.821801    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z" event={"ID":"ec1f670e-3d8f-415c-b1ce-51e48eb96c8b","Type":"ContainerDied","Data":"f92916f318724a016647c3da49e97c71818c928a2e9ac7bd7316d24636394951"}
Sep 10 14:52:47.822334 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.821824    2560 scope.go:117] "RemoveContainer" containerID="17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a"
Sep 10 14:52:47.831443 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.831425    2560 scope.go:117] "RemoveContainer" containerID="17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a"
Sep 10 14:52:47.831683 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:52:47.831664    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a\": container with ID starting with 17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a not found: ID does not exist" containerID="17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a"
Sep 10 14:52:47.831739 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.831691    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a"} err="failed to get container status \"17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a\": rpc error: code = NotFound desc = could not find container \"17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a\": container with ID starting with 17744c0f41a6803c4ee0ce4dd2564be244d8271765e6cf9139a7918695bfaf2a not found: ID does not exist"
Sep 10 14:52:47.840606 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.840579    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"]
Sep 10 14:52:47.844466 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:47.844444    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6eb765-8489549886-x8d4z"]
Sep 10 14:52:49.319910 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:49.319877    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" path="/var/lib/kubelet/pods/ec1f670e-3d8f-415c-b1ce-51e48eb96c8b/volumes"
Sep 10 14:52:51.950732 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:51.950697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:53.442264 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:53.442233    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:53.542901 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:53.542863    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:53.695860 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:53.695784    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:54.193991 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:54.193960    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:58.827467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:58.827434    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"]
Sep 10 14:52:58.840398 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:52:58.840370    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:00.410528 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:00.410487    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"]
Sep 10 14:53:00.429408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:00.429366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:07.286028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:07.285992    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:53:08.654866 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:08.654828    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:09.502756 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.502719    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:09.535093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.535057    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"]
Sep 10 14:53:09.535338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.535309    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk" podUID="5ecb0487-f599-4a1d-9328-93f92aa7030e" containerName="payload-pre-processing" containerID="cri-o://b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd" gracePeriod=30
Sep 10 14:53:09.603119 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.603083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:09.685984 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.685944    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:09.743808 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.743766    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:09.806544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.806518    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:53:09.902527 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.902492    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlf7w\" (UniqueName: \"kubernetes.io/projected/5ecb0487-f599-4a1d-9328-93f92aa7030e-kube-api-access-vlf7w\") pod \"5ecb0487-f599-4a1d-9328-93f92aa7030e\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") "
Sep 10 14:53:09.902697 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.902547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ecb0487-f599-4a1d-9328-93f92aa7030e-plugins-config-volume\") pod \"5ecb0487-f599-4a1d-9328-93f92aa7030e\" (UID: \"5ecb0487-f599-4a1d-9328-93f92aa7030e\") "
Sep 10 14:53:09.902977 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.902954    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ecb0487-f599-4a1d-9328-93f92aa7030e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ecb0487-f599-4a1d-9328-93f92aa7030e" (UID: "5ecb0487-f599-4a1d-9328-93f92aa7030e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:53:09.904767 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.904746    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ecb0487-f599-4a1d-9328-93f92aa7030e-kube-api-access-vlf7w" (OuterVolumeSpecName: "kube-api-access-vlf7w") pod "5ecb0487-f599-4a1d-9328-93f92aa7030e" (UID: "5ecb0487-f599-4a1d-9328-93f92aa7030e"). InnerVolumeSpecName "kube-api-access-vlf7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:53:09.971380 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.971349    2560 generic.go:356] "Generic (PLEG): container finished" podID="5ecb0487-f599-4a1d-9328-93f92aa7030e" containerID="b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd" exitCode=0
Sep 10 14:53:09.971550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.971393    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"
Sep 10 14:53:09.971550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.971431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk" event={"ID":"5ecb0487-f599-4a1d-9328-93f92aa7030e","Type":"ContainerDied","Data":"b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd"}
Sep 10 14:53:09.971550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.971478    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk" event={"ID":"5ecb0487-f599-4a1d-9328-93f92aa7030e","Type":"ContainerDied","Data":"016537d22c8b16571aec13b0bc936bd14256644dce3eaf334a923f45c7716bf0"}
Sep 10 14:53:09.971550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.971501    2560 scope.go:117] "RemoveContainer" containerID="b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd"
Sep 10 14:53:09.981408 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.981389    2560 scope.go:117] "RemoveContainer" containerID="b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd"
Sep 10 14:53:09.981677 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:53:09.981659    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd\": container with ID starting with b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd not found: ID does not exist" containerID="b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd"
Sep 10 14:53:09.981721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:09.981686    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd"} err="failed to get container status \"b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd\": rpc error: code = NotFound desc = could not find container \"b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd\": container with ID starting with b15de751a3154ce4f1caea0e5e9df272fc8eb34ba0d65d36020ce6ffc127fbbd not found: ID does not exist"
Sep 10 14:53:10.004018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:10.003991    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlf7w\" (UniqueName: \"kubernetes.io/projected/5ecb0487-f599-4a1d-9328-93f92aa7030e-kube-api-access-vlf7w\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:10.004018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:10.004014    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ecb0487-f599-4a1d-9328-93f92aa7030e-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:10.009487 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:10.009461    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"]
Sep 10 14:53:10.015721 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:10.015695    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b717-5bcd9794f8-j44xk"]
Sep 10 14:53:10.063054 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:10.062955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:11.319397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:11.319366    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ecb0487-f599-4a1d-9328-93f92aa7030e" path="/var/lib/kubelet/pods/5ecb0487-f599-4a1d-9328-93f92aa7030e/volumes"
Sep 10 14:53:13.848898 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:13.848795    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd" podUID="7bd58eb8-75c8-462f-bf2e-58919503e4a4" containerName="main" containerID="cri-o://bd522403d9380acf74997870dd8cb76c802778344e16915e41bfaf23a3d11b1c" gracePeriod=45
Sep 10 14:53:14.001976 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.001942    2560 generic.go:356] "Generic (PLEG): container finished" podID="7bd58eb8-75c8-462f-bf2e-58919503e4a4" containerID="bd522403d9380acf74997870dd8cb76c802778344e16915e41bfaf23a3d11b1c" exitCode=0
Sep 10 14:53:14.001976 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.001968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd" event={"ID":"7bd58eb8-75c8-462f-bf2e-58919503e4a4","Type":"ContainerDied","Data":"bd522403d9380acf74997870dd8cb76c802778344e16915e41bfaf23a3d11b1c"}
Sep 10 14:53:14.121806 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.121777    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:53:14.245220 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245187    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgvtn\" (UniqueName: \"kubernetes.io/projected/7bd58eb8-75c8-462f-bf2e-58919503e4a4-kube-api-access-vgvtn\") pod \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") "
Sep 10 14:53:14.245399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245239    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tmp-dir\") pod \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") "
Sep 10 14:53:14.245399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245361    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-model-cache\") pod \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") "
Sep 10 14:53:14.245483 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245405    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-dshm\") pod \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") "
Sep 10 14:53:14.245483 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245431    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tls-certs\") pod \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") "
Sep 10 14:53:14.245580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245488    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-home\") pod \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\" (UID: \"7bd58eb8-75c8-462f-bf2e-58919503e4a4\") "
Sep 10 14:53:14.245580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245559    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7bd58eb8-75c8-462f-bf2e-58919503e4a4" (UID: "7bd58eb8-75c8-462f-bf2e-58919503e4a4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:14.245690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245648    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-model-cache" (OuterVolumeSpecName: "model-cache") pod "7bd58eb8-75c8-462f-bf2e-58919503e4a4" (UID: "7bd58eb8-75c8-462f-bf2e-58919503e4a4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:14.245856 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245829    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-home" (OuterVolumeSpecName: "home") pod "7bd58eb8-75c8-462f-bf2e-58919503e4a4" (UID: "7bd58eb8-75c8-462f-bf2e-58919503e4a4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:14.245992 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245924    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:14.245992 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245937    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:14.245992 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.245948    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:14.248271 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.248214    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-dshm" (OuterVolumeSpecName: "dshm") pod "7bd58eb8-75c8-462f-bf2e-58919503e4a4" (UID: "7bd58eb8-75c8-462f-bf2e-58919503e4a4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:14.248466 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.248439    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7bd58eb8-75c8-462f-bf2e-58919503e4a4" (UID: "7bd58eb8-75c8-462f-bf2e-58919503e4a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:53:14.248532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.248465    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd58eb8-75c8-462f-bf2e-58919503e4a4-kube-api-access-vgvtn" (OuterVolumeSpecName: "kube-api-access-vgvtn") pod "7bd58eb8-75c8-462f-bf2e-58919503e4a4" (UID: "7bd58eb8-75c8-462f-bf2e-58919503e4a4"). InnerVolumeSpecName "kube-api-access-vgvtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:53:14.347340 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.347270    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgvtn\" (UniqueName: \"kubernetes.io/projected/7bd58eb8-75c8-462f-bf2e-58919503e4a4-kube-api-access-vgvtn\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:14.347340 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.347335    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bd58eb8-75c8-462f-bf2e-58919503e4a4-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:14.347595 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:14.347353    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bd58eb8-75c8-462f-bf2e-58919503e4a4-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:15.009799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.009764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd" event={"ID":"7bd58eb8-75c8-462f-bf2e-58919503e4a4","Type":"ContainerDied","Data":"c38e981f1028f82308448a332b3490eafe523a336403cc6442996d58e4d4b79b"}
Sep 10 14:53:15.009799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.009797    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"
Sep 10 14:53:15.010314 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.009815    2560 scope.go:117] "RemoveContainer" containerID="bd522403d9380acf74997870dd8cb76c802778344e16915e41bfaf23a3d11b1c"
Sep 10 14:53:15.030078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.030050    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"]
Sep 10 14:53:15.032789 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.032764    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-b-f6cac41e-kserve-7cbdb85944-j2cjd"]
Sep 10 14:53:15.320153 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.320075    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bd58eb8-75c8-462f-bf2e-58919503e4a4" path="/var/lib/kubelet/pods/7bd58eb8-75c8-462f-bf2e-58919503e4a4/volumes"
Sep 10 14:53:15.434518 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.434479    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46" podUID="e90498b0-6417-4d3b-bd43-f5ed29eca51b" containerName="main" containerID="cri-o://4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763" gracePeriod=45
Sep 10 14:53:15.673670 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.673637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:15.708292 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.708249    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:53:15.761720 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.761683    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmchc\" (UniqueName: \"kubernetes.io/projected/e90498b0-6417-4d3b-bd43-f5ed29eca51b-kube-api-access-nmchc\") pod \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") "
Sep 10 14:53:15.761882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.761733    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tls-certs\") pod \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") "
Sep 10 14:53:15.761882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.761775    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tmp-dir\") pod \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") "
Sep 10 14:53:15.761882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.761797    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-home\") pod \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") "
Sep 10 14:53:15.761882 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.761868    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-model-cache\") pod \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") "
Sep 10 14:53:15.762098 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.761895    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-dshm\") pod \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\" (UID: \"e90498b0-6417-4d3b-bd43-f5ed29eca51b\") "
Sep 10 14:53:15.762158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.762085    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e90498b0-6417-4d3b-bd43-f5ed29eca51b" (UID: "e90498b0-6417-4d3b-bd43-f5ed29eca51b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:15.762158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.762142    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-home" (OuterVolumeSpecName: "home") pod "e90498b0-6417-4d3b-bd43-f5ed29eca51b" (UID: "e90498b0-6417-4d3b-bd43-f5ed29eca51b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:15.762388 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.762366    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:15.762388 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.762376    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-model-cache" (OuterVolumeSpecName: "model-cache") pod "e90498b0-6417-4d3b-bd43-f5ed29eca51b" (UID: "e90498b0-6417-4d3b-bd43-f5ed29eca51b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:15.762580 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.762388    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:15.764018 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.763990    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e90498b0-6417-4d3b-bd43-f5ed29eca51b-kube-api-access-nmchc" (OuterVolumeSpecName: "kube-api-access-nmchc") pod "e90498b0-6417-4d3b-bd43-f5ed29eca51b" (UID: "e90498b0-6417-4d3b-bd43-f5ed29eca51b"). InnerVolumeSpecName "kube-api-access-nmchc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:53:15.764110 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.764032    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-dshm" (OuterVolumeSpecName: "dshm") pod "e90498b0-6417-4d3b-bd43-f5ed29eca51b" (UID: "e90498b0-6417-4d3b-bd43-f5ed29eca51b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:53:15.764359 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.764341    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e90498b0-6417-4d3b-bd43-f5ed29eca51b" (UID: "e90498b0-6417-4d3b-bd43-f5ed29eca51b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:53:15.863459 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.863363    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:15.863459 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.863406    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e90498b0-6417-4d3b-bd43-f5ed29eca51b-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:15.863459 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.863416    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmchc\" (UniqueName: \"kubernetes.io/projected/e90498b0-6417-4d3b-bd43-f5ed29eca51b-kube-api-access-nmchc\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:15.863459 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:15.863426    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e90498b0-6417-4d3b-bd43-f5ed29eca51b-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:16.018042 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.018003    2560 generic.go:356] "Generic (PLEG): container finished" podID="e90498b0-6417-4d3b-bd43-f5ed29eca51b" containerID="4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763" exitCode=0
Sep 10 14:53:16.018465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.018049    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"
Sep 10 14:53:16.018465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.018097    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46" event={"ID":"e90498b0-6417-4d3b-bd43-f5ed29eca51b","Type":"ContainerDied","Data":"4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763"}
Sep 10 14:53:16.018465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.018136    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46" event={"ID":"e90498b0-6417-4d3b-bd43-f5ed29eca51b","Type":"ContainerDied","Data":"8b144a161836094b7fd9a387e95531d28c3ad7ab5f822ba9142cfe00254029e0"}
Sep 10 14:53:16.018465 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.018156    2560 scope.go:117] "RemoveContainer" containerID="4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763"
Sep 10 14:53:16.028138 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.028120    2560 scope.go:117] "RemoveContainer" containerID="4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763"
Sep 10 14:53:16.028414 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:53:16.028394    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763\": container with ID starting with 4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763 not found: ID does not exist" containerID="4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763"
Sep 10 14:53:16.028466 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.028428    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763"} err="failed to get container status \"4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763\": rpc error: code = NotFound desc = could not find container \"4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763\": container with ID starting with 4db3c9d627d6177ea673b26487eaf45d0241445a81dc9bef8048fb6c8f8a0763 not found: ID does not exist"
Sep 10 14:53:16.040163 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.040139    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"]
Sep 10 14:53:16.043644 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:16.043624    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/e2e-conflict-a-f6cac41e-kserve-9855f8cd6-lvh46"]
Sep 10 14:53:17.142390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:17.142356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:17.320186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:17.320153    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e90498b0-6417-4d3b-bd43-f5ed29eca51b" path="/var/lib/kubelet/pods/e90498b0-6417-4d3b-bd43-f5ed29eca51b/volumes"
Sep 10 14:53:20.510926 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:20.510897    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:26.453101 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:26.453071    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:27.041484 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.041447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:27.073630 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.073593    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"]
Sep 10 14:53:27.073919 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.073864    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh" podUID="f2be7141-d1bb-4a32-9218-f36497842ad5" containerName="payload-processing" containerID="cri-o://4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12" gracePeriod=30
Sep 10 14:53:27.082215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.082189    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"]
Sep 10 14:53:27.082966 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.082928    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722" podUID="41064590-4693-429b-9892-eefc37ce1a68" containerName="payload-pre-processing" containerID="cri-o://bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492" gracePeriod=30
Sep 10 14:53:27.108953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.108903    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:27.169999 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.169965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:27.230150 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.230108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:27.289667 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.289618    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:27.340309 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.340267    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:53:27.374647 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.374615    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:53:27.489479 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.489440    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r86cm\" (UniqueName: \"kubernetes.io/projected/41064590-4693-429b-9892-eefc37ce1a68-kube-api-access-r86cm\") pod \"41064590-4693-429b-9892-eefc37ce1a68\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") "
Sep 10 14:53:27.489945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.489508    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41064590-4693-429b-9892-eefc37ce1a68-plugins-config-volume\") pod \"41064590-4693-429b-9892-eefc37ce1a68\" (UID: \"41064590-4693-429b-9892-eefc37ce1a68\") "
Sep 10 14:53:27.489945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.489588    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2be7141-d1bb-4a32-9218-f36497842ad5-plugins-config-volume\") pod \"f2be7141-d1bb-4a32-9218-f36497842ad5\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") "
Sep 10 14:53:27.489945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.489683    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lndtp\" (UniqueName: \"kubernetes.io/projected/f2be7141-d1bb-4a32-9218-f36497842ad5-kube-api-access-lndtp\") pod \"f2be7141-d1bb-4a32-9218-f36497842ad5\" (UID: \"f2be7141-d1bb-4a32-9218-f36497842ad5\") "
Sep 10 14:53:27.489945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.489904    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41064590-4693-429b-9892-eefc37ce1a68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41064590-4693-429b-9892-eefc37ce1a68" (UID: "41064590-4693-429b-9892-eefc37ce1a68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:53:27.489945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.489915    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2be7141-d1bb-4a32-9218-f36497842ad5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2be7141-d1bb-4a32-9218-f36497842ad5" (UID: "f2be7141-d1bb-4a32-9218-f36497842ad5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:53:27.490127 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.490093    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41064590-4693-429b-9892-eefc37ce1a68-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:27.490127 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.490107    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2be7141-d1bb-4a32-9218-f36497842ad5-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:27.491666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.491644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41064590-4693-429b-9892-eefc37ce1a68-kube-api-access-r86cm" (OuterVolumeSpecName: "kube-api-access-r86cm") pod "41064590-4693-429b-9892-eefc37ce1a68" (UID: "41064590-4693-429b-9892-eefc37ce1a68"). InnerVolumeSpecName "kube-api-access-r86cm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:53:27.491666 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.491652    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2be7141-d1bb-4a32-9218-f36497842ad5-kube-api-access-lndtp" (OuterVolumeSpecName: "kube-api-access-lndtp") pod "f2be7141-d1bb-4a32-9218-f36497842ad5" (UID: "f2be7141-d1bb-4a32-9218-f36497842ad5"). InnerVolumeSpecName "kube-api-access-lndtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:53:27.591070 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.590990    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r86cm\" (UniqueName: \"kubernetes.io/projected/41064590-4693-429b-9892-eefc37ce1a68-kube-api-access-r86cm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:27.591070 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.591019    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lndtp\" (UniqueName: \"kubernetes.io/projected/f2be7141-d1bb-4a32-9218-f36497842ad5-kube-api-access-lndtp\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:27.593719 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:27.593695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:28.096673 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.096640    2560 generic.go:356] "Generic (PLEG): container finished" podID="f2be7141-d1bb-4a32-9218-f36497842ad5" containerID="4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12" exitCode=0
Sep 10 14:53:28.096790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.096695    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"
Sep 10 14:53:28.096790 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.096723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh" event={"ID":"f2be7141-d1bb-4a32-9218-f36497842ad5","Type":"ContainerDied","Data":"4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12"}
Sep 10 14:53:28.096912 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.096814    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh" event={"ID":"f2be7141-d1bb-4a32-9218-f36497842ad5","Type":"ContainerDied","Data":"39a12e67c2e1737930ae1480dca941cc5f3c6e63cbbfeed38c463e6b5f2d56cc"}
Sep 10 14:53:28.096912 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.096843    2560 scope.go:117] "RemoveContainer" containerID="4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12"
Sep 10 14:53:28.098450 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.098429    2560 generic.go:356] "Generic (PLEG): container finished" podID="41064590-4693-429b-9892-eefc37ce1a68" containerID="bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492" exitCode=0
Sep 10 14:53:28.098544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.098491    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722" event={"ID":"41064590-4693-429b-9892-eefc37ce1a68","Type":"ContainerDied","Data":"bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492"}
Sep 10 14:53:28.098544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.098499    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"
Sep 10 14:53:28.098544 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.098533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722" event={"ID":"41064590-4693-429b-9892-eefc37ce1a68","Type":"ContainerDied","Data":"86c53154c9a358d131a3fbc41da32cdf5c8d45b8aea2855d24b98dd93d0bd436"}
Sep 10 14:53:28.109550 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.109524    2560 scope.go:117] "RemoveContainer" containerID="4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12"
Sep 10 14:53:28.109837 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:53:28.109819    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12\": container with ID starting with 4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12 not found: ID does not exist" containerID="4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12"
Sep 10 14:53:28.109909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.109845    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12"} err="failed to get container status \"4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12\": rpc error: code = NotFound desc = could not find container \"4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12\": container with ID starting with 4cb2d19c878d5c577098db3d3dc20ee3f60f794487d86b983a4d0fcf5f329c12 not found: ID does not exist"
Sep 10 14:53:28.109909 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.109860    2560 scope.go:117] "RemoveContainer" containerID="bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492"
Sep 10 14:53:28.119686 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.119665    2560 scope.go:117] "RemoveContainer" containerID="bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492"
Sep 10 14:53:28.119971 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:53:28.119947    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492\": container with ID starting with bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492 not found: ID does not exist" containerID="bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492"
Sep 10 14:53:28.120030 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.119983    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492"} err="failed to get container status \"bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492\": rpc error: code = NotFound desc = could not find container \"bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492\": container with ID starting with bd2735b10846a668440dab8e29591f2417e06bc8887eab74ac242987c953f492 not found: ID does not exist"
Sep 10 14:53:28.121617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.121594    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"]
Sep 10 14:53:28.124328 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.124308    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5c9641-5b7974f47c-bk6qh"]
Sep 10 14:53:28.131787 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.131765    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"]
Sep 10 14:53:28.135816 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:28.135796    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5c9641-7567bbc886-hf722"]
Sep 10 14:53:29.320689 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:29.320655    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41064590-4693-429b-9892-eefc37ce1a68" path="/var/lib/kubelet/pods/41064590-4693-429b-9892-eefc37ce1a68/volumes"
Sep 10 14:53:29.321078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:29.321037    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2be7141-d1bb-4a32-9218-f36497842ad5" path="/var/lib/kubelet/pods/f2be7141-d1bb-4a32-9218-f36497842ad5/volumes"
Sep 10 14:53:33.211894 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:33.211858    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:33.271031 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:33.270996    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:33.480131 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:33.480031    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:34.023312 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:34.023264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:44.311181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.311137    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"]
Sep 10 14:53:44.311614 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.311408    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" podUID="2c76b901-4e73-4e90-8b9b-56cdb0263ba9" containerName="maas-api" containerID="cri-o://be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566" gracePeriod=30
Sep 10 14:53:44.333903 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.333875    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:44.403791 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.403755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:44.505497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.505465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:44.561477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.561415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:44.568979 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.568960    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:53:44.665160 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.665113    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-openshift-service-ca\") pod \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") "
Sep 10 14:53:44.665354 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.665222    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") pod \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") "
Sep 10 14:53:44.665354 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.665260    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-metrics-certs\") pod \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") "
Sep 10 14:53:44.665450 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.665352    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw2w2\" (UniqueName: \"kubernetes.io/projected/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-kube-api-access-rw2w2\") pod \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\" (UID: \"2c76b901-4e73-4e90-8b9b-56cdb0263ba9\") "
Sep 10 14:53:44.665560 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.665529    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c76b901-4e73-4e90-8b9b-56cdb0263ba9" (UID: "2c76b901-4e73-4e90-8b9b-56cdb0263ba9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:53:44.665799 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.665775    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:44.667480 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.667453    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c76b901-4e73-4e90-8b9b-56cdb0263ba9" (UID: "2c76b901-4e73-4e90-8b9b-56cdb0263ba9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:53:44.667575 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.667550    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-kube-api-access-rw2w2" (OuterVolumeSpecName: "kube-api-access-rw2w2") pod "2c76b901-4e73-4e90-8b9b-56cdb0263ba9" (UID: "2c76b901-4e73-4e90-8b9b-56cdb0263ba9"). InnerVolumeSpecName "kube-api-access-rw2w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:53:44.667791 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.667772    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2c76b901-4e73-4e90-8b9b-56cdb0263ba9" (UID: "2c76b901-4e73-4e90-8b9b-56cdb0263ba9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:53:44.767103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.767066    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:44.767103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.767095    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:44.767103 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.767104    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw2w2\" (UniqueName: \"kubernetes.io/projected/2c76b901-4e73-4e90-8b9b-56cdb0263ba9-kube-api-access-rw2w2\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:53:44.908516 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:44.908431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:45.203737 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.203700    2560 generic.go:356] "Generic (PLEG): container finished" podID="2c76b901-4e73-4e90-8b9b-56cdb0263ba9" containerID="be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566" exitCode=0
Sep 10 14:53:45.203915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.203768    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"
Sep 10 14:53:45.203915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.203794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" event={"ID":"2c76b901-4e73-4e90-8b9b-56cdb0263ba9","Type":"ContainerDied","Data":"be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566"}
Sep 10 14:53:45.203915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.203873    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6" event={"ID":"2c76b901-4e73-4e90-8b9b-56cdb0263ba9","Type":"ContainerDied","Data":"6afe9d21896a1b89825d91f3483069cfdcbe6ea995765f75b3385b63385f893d"}
Sep 10 14:53:45.203915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.203899    2560 scope.go:117] "RemoveContainer" containerID="be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566"
Sep 10 14:53:45.214120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.214096    2560 scope.go:117] "RemoveContainer" containerID="be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566"
Sep 10 14:53:45.214449 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:53:45.214429    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566\": container with ID starting with be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566 not found: ID does not exist" containerID="be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566"
Sep 10 14:53:45.214529 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.214458    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566"} err="failed to get container status \"be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566\": rpc error: code = NotFound desc = could not find container \"be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566\": container with ID starting with be53912f546081d181afbceaf42dcedfe52f8226f3ee9e84c4882bc70c62d566 not found: ID does not exist"
Sep 10 14:53:45.224680 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.224656    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"]
Sep 10 14:53:45.229446 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.229422    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecfa25-79b4f644fd-f92f6"]
Sep 10 14:53:45.319390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:45.319350    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c76b901-4e73-4e90-8b9b-56cdb0263ba9" path="/var/lib/kubelet/pods/2c76b901-4e73-4e90-8b9b-56cdb0263ba9/volumes"
Sep 10 14:53:50.751093 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:50.751056    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:50.850072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:50.850029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:51.029061 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:53:51.028973    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:00.029448 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:54:00.029417    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:14.363777 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:54:14.363719    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:25.779627 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:54:25.779583    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:36.482842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:54:36.482806    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:55:24.997940 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:24.997903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:55:48.031534 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:48.031480    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:48.213199 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:48.213164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:48.380366 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:48.380244    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:48.578413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:48.578357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:48.749731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:48.749697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:48.979747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:48.979688    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:49.121086 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:49.120982    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:49.434792 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:49.434694    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:49.582142 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:49.582111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:50.216904 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:50.216872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:50.321039 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:50.321008    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:50.788341 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:50.788274    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:51.015407 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:51.015368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:52.352509 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:52.352475    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:52.851089 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:52.851039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:54.500501 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:54.500459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:54.803149 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:54.803059    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:55.095559 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:55.095435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:55.333149 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:55.333117    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:55.984000 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:55.983963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:56.207390 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:56.207355    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:56.651252 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:56.651220    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:56.885447 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:56.885412    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:57.188341 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:57.188261    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:57.341950 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:57.341914    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"]
Sep 10 14:55:57.419630 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:57.419595    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"]
Sep 10 14:55:57.451076 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:57.450994    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"]
Sep 10 14:55:57.696525 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:57.696492    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:57.924757 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:57.924673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:59.372145 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:55:59.372096    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:56:09.011837 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.011800    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"]
Sep 10 14:56:09.012268 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.012011    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" podUID="170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" containerName="maas-api" containerID="cri-o://aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130" gracePeriod=30
Sep 10 14:56:09.127328 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.127296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:09.284072 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.284041    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"]
Sep 10 14:56:09.284556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.284372    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl" podUID="2116c427-8cb4-4383-95ce-4386943e5546" containerName="payload-processing" containerID="cri-o://9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb" gracePeriod=30
Sep 10 14:56:09.284556 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.284451    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:56:09.375575 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.375540    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls\") pod \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") "
Sep 10 14:56:09.375747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.375587    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-metrics-certs\") pod \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") "
Sep 10 14:56:09.375747 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.375691    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-openshift-service-ca\") pod \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") "
Sep 10 14:56:09.375854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.375784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4sch\" (UniqueName: \"kubernetes.io/projected/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-kube-api-access-t4sch\") pod \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\" (UID: \"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a\") "
Sep 10 14:56:09.376130 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.376088    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" (UID: "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:56:09.377853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.377820    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" (UID: "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:09.377945 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.377868    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" (UID: "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:09.378203 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.378181    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-kube-api-access-t4sch" (OuterVolumeSpecName: "kube-api-access-t4sch") pod "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" (UID: "170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a"). InnerVolumeSpecName "kube-api-access-t4sch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:09.443272 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.443241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:09.476975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.476944    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:09.476975 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.476970    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:09.477176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.476981    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:09.477176 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.476997    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4sch\" (UniqueName: \"kubernetes.io/projected/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a-kube-api-access-t4sch\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:09.540045 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.540021    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:56:09.600207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.600160    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:09.678983 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.678941    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2116c427-8cb4-4383-95ce-4386943e5546-plugins-config-volume\") pod \"2116c427-8cb4-4383-95ce-4386943e5546\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") "
Sep 10 14:56:09.679179 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.679091    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpjsv\" (UniqueName: \"kubernetes.io/projected/2116c427-8cb4-4383-95ce-4386943e5546-kube-api-access-vpjsv\") pod \"2116c427-8cb4-4383-95ce-4386943e5546\" (UID: \"2116c427-8cb4-4383-95ce-4386943e5546\") "
Sep 10 14:56:09.679392 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.679366    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2116c427-8cb4-4383-95ce-4386943e5546-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2116c427-8cb4-4383-95ce-4386943e5546" (UID: "2116c427-8cb4-4383-95ce-4386943e5546"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:56:09.679477 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.679458    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2116c427-8cb4-4383-95ce-4386943e5546-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:09.681235 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.681213    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2116c427-8cb4-4383-95ce-4386943e5546-kube-api-access-vpjsv" (OuterVolumeSpecName: "kube-api-access-vpjsv") pod "2116c427-8cb4-4383-95ce-4386943e5546" (UID: "2116c427-8cb4-4383-95ce-4386943e5546"). InnerVolumeSpecName "kube-api-access-vpjsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:09.780338 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.780304    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpjsv\" (UniqueName: \"kubernetes.io/projected/2116c427-8cb4-4383-95ce-4386943e5546-kube-api-access-vpjsv\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:09.861115 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:09.861028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:10.065349 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.065314    2560 generic.go:356] "Generic (PLEG): container finished" podID="170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" containerID="aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130" exitCode=0
Sep 10 14:56:10.065795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.065357    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"
Sep 10 14:56:10.065795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.065394    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" event={"ID":"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a","Type":"ContainerDied","Data":"aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130"}
Sep 10 14:56:10.065795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.065443    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j" event={"ID":"170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a","Type":"ContainerDied","Data":"9807f9723083f80099b84593037c208c3a5b51146370444130e4005ff9a6eea8"}
Sep 10 14:56:10.065795 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.065473    2560 scope.go:117] "RemoveContainer" containerID="aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130"
Sep 10 14:56:10.067126 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.067101    2560 generic.go:356] "Generic (PLEG): container finished" podID="2116c427-8cb4-4383-95ce-4386943e5546" containerID="9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb" exitCode=0
Sep 10 14:56:10.067218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.067147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl" event={"ID":"2116c427-8cb4-4383-95ce-4386943e5546","Type":"ContainerDied","Data":"9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb"}
Sep 10 14:56:10.067218 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.067188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl" event={"ID":"2116c427-8cb4-4383-95ce-4386943e5546","Type":"ContainerDied","Data":"3fe0507c22df9bc4ebaa0e14f2c38045e3e739bc2cec11aa7db07026fa29dc55"}
Sep 10 14:56:10.067337 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.067217    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"
Sep 10 14:56:10.077007 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.076933    2560 scope.go:117] "RemoveContainer" containerID="aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130"
Sep 10 14:56:10.077314 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:10.077267    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130\": container with ID starting with aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130 not found: ID does not exist" containerID="aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130"
Sep 10 14:56:10.077416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.077325    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130"} err="failed to get container status \"aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130\": rpc error: code = NotFound desc = could not find container \"aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130\": container with ID starting with aa07d1d7497a3d75cca9a6507cd7398152a8c785610dc4c5f63da6de004ea130 not found: ID does not exist"
Sep 10 14:56:10.077416 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.077348    2560 scope.go:117] "RemoveContainer" containerID="9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb"
Sep 10 14:56:10.086063 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.086046    2560 scope.go:117] "RemoveContainer" containerID="9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb"
Sep 10 14:56:10.086363 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:10.086342    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb\": container with ID starting with 9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb not found: ID does not exist" containerID="9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb"
Sep 10 14:56:10.086429 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.086371    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb"} err="failed to get container status \"9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb\": rpc error: code = NotFound desc = could not find container \"9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb\": container with ID starting with 9be4957541d0aa9b2ec80ca9509ffec2fe64e841e37e161916451735f57620fb not found: ID does not exist"
Sep 10 14:56:10.117158 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.117086    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"]
Sep 10 14:56:10.119552 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.119531    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9b1e67-75bb659599-sgfrl"]
Sep 10 14:56:10.140009 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.139974    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"]
Sep 10 14:56:10.148783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.148746    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9b1e67-5ccf6b69fd-4n49j"]
Sep 10 14:56:10.406344 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:10.406236    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:11.319330 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:11.319271    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" path="/var/lib/kubelet/pods/170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a/volumes"
Sep 10 14:56:11.319731 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:11.319713    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2116c427-8cb4-4383-95ce-4386943e5546" path="/var/lib/kubelet/pods/2116c427-8cb4-4383-95ce-4386943e5546/volumes"
Sep 10 14:56:12.362255 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.362216    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc" podUID="585bbeb6-0249-433c-a2cd-7bcfb3823259" containerName="main" containerID="cri-o://30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c" gracePeriod=45
Sep 10 14:56:12.440006 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.439968    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc" podUID="136ceb19-3aaf-4d94-beba-1dc83e8fcbab" containerName="main" containerID="cri-o://316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc" gracePeriod=45
Sep 10 14:56:12.470650 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.470612    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2" podUID="9bb9602d-68d3-40c2-8561-805c2014e908" containerName="main" containerID="cri-o://eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f" gracePeriod=45
Sep 10 14:56:12.705623 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.705205    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:56:12.755757 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.755728    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:56:12.770417 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.770391    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:56:12.814133 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814098    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-dshm\") pod \"585bbeb6-0249-433c-a2cd-7bcfb3823259\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") "
Sep 10 14:56:12.814332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814166    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-home\") pod \"585bbeb6-0249-433c-a2cd-7bcfb3823259\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") "
Sep 10 14:56:12.814332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-tmp-dir\") pod \"585bbeb6-0249-433c-a2cd-7bcfb3823259\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") "
Sep 10 14:56:12.814332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814267    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bbeb6-0249-433c-a2cd-7bcfb3823259-tls-certs\") pod \"585bbeb6-0249-433c-a2cd-7bcfb3823259\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") "
Sep 10 14:56:12.814332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814311    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-model-cache\") pod \"585bbeb6-0249-433c-a2cd-7bcfb3823259\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") "
Sep 10 14:56:12.814569 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814365    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zrwp\" (UniqueName: \"kubernetes.io/projected/585bbeb6-0249-433c-a2cd-7bcfb3823259-kube-api-access-4zrwp\") pod \"585bbeb6-0249-433c-a2cd-7bcfb3823259\" (UID: \"585bbeb6-0249-433c-a2cd-7bcfb3823259\") "
Sep 10 14:56:12.814855 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814724    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "585bbeb6-0249-433c-a2cd-7bcfb3823259" (UID: "585bbeb6-0249-433c-a2cd-7bcfb3823259"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.814855 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814823    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-home" (OuterVolumeSpecName: "home") pod "585bbeb6-0249-433c-a2cd-7bcfb3823259" (UID: "585bbeb6-0249-433c-a2cd-7bcfb3823259"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.815086 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.814951    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-model-cache" (OuterVolumeSpecName: "model-cache") pod "585bbeb6-0249-433c-a2cd-7bcfb3823259" (UID: "585bbeb6-0249-433c-a2cd-7bcfb3823259"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.817151 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.817114    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585bbeb6-0249-433c-a2cd-7bcfb3823259-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "585bbeb6-0249-433c-a2cd-7bcfb3823259" (UID: "585bbeb6-0249-433c-a2cd-7bcfb3823259"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:12.817267 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.817122    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-dshm" (OuterVolumeSpecName: "dshm") pod "585bbeb6-0249-433c-a2cd-7bcfb3823259" (UID: "585bbeb6-0249-433c-a2cd-7bcfb3823259"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.817267 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.817242    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585bbeb6-0249-433c-a2cd-7bcfb3823259-kube-api-access-4zrwp" (OuterVolumeSpecName: "kube-api-access-4zrwp") pod "585bbeb6-0249-433c-a2cd-7bcfb3823259" (UID: "585bbeb6-0249-433c-a2cd-7bcfb3823259"). InnerVolumeSpecName "kube-api-access-4zrwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:12.915927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.915849    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4gkr\" (UniqueName: \"kubernetes.io/projected/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-kube-api-access-g4gkr\") pod \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") "
Sep 10 14:56:12.915927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.915885    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-home\") pod \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") "
Sep 10 14:56:12.915927 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.915921    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tls-certs\") pod \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.915943    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tmp-dir\") pod \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916054    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv2wr\" (UniqueName: \"kubernetes.io/projected/9bb9602d-68d3-40c2-8561-805c2014e908-kube-api-access-qv2wr\") pod \"9bb9602d-68d3-40c2-8561-805c2014e908\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916088    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-dshm\") pod \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916112    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-dshm\") pod \"9bb9602d-68d3-40c2-8561-805c2014e908\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916148    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb9602d-68d3-40c2-8561-805c2014e908-tls-certs\") pod \"9bb9602d-68d3-40c2-8561-805c2014e908\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916159    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-home" (OuterVolumeSpecName: "home") pod "136ceb19-3aaf-4d94-beba-1dc83e8fcbab" (UID: "136ceb19-3aaf-4d94-beba-1dc83e8fcbab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916181    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-tmp-dir\") pod \"9bb9602d-68d3-40c2-8561-805c2014e908\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") "
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916191    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "136ceb19-3aaf-4d94-beba-1dc83e8fcbab" (UID: "136ceb19-3aaf-4d94-beba-1dc83e8fcbab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.916215 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916212    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-home\") pod \"9bb9602d-68d3-40c2-8561-805c2014e908\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") "
Sep 10 14:56:12.916730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916240    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-model-cache\") pod \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\" (UID: \"136ceb19-3aaf-4d94-beba-1dc83e8fcbab\") "
Sep 10 14:56:12.916730 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916268    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-model-cache\") pod \"9bb9602d-68d3-40c2-8561-805c2014e908\" (UID: \"9bb9602d-68d3-40c2-8561-805c2014e908\") "
Sep 10 14:56:12.916832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916768    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.916832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916790    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zrwp\" (UniqueName: \"kubernetes.io/projected/585bbeb6-0249-433c-a2cd-7bcfb3823259-kube-api-access-4zrwp\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.916832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916802    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.916832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916816    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.916832 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916813    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-home" (OuterVolumeSpecName: "home") pod "9bb9602d-68d3-40c2-8561-805c2014e908" (UID: "9bb9602d-68d3-40c2-8561-805c2014e908"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.917081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916828    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.917081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916864    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.917081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916883    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bbeb6-0249-433c-a2cd-7bcfb3823259-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.917081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.916898    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bbeb6-0249-433c-a2cd-7bcfb3823259-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:12.917081 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.917012    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-model-cache" (OuterVolumeSpecName: "model-cache") pod "9bb9602d-68d3-40c2-8561-805c2014e908" (UID: "9bb9602d-68d3-40c2-8561-805c2014e908"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.917347 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.917099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9bb9602d-68d3-40c2-8561-805c2014e908" (UID: "9bb9602d-68d3-40c2-8561-805c2014e908"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.917347 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.917223    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-model-cache" (OuterVolumeSpecName: "model-cache") pod "136ceb19-3aaf-4d94-beba-1dc83e8fcbab" (UID: "136ceb19-3aaf-4d94-beba-1dc83e8fcbab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.918515 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.918487    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-dshm" (OuterVolumeSpecName: "dshm") pod "9bb9602d-68d3-40c2-8561-805c2014e908" (UID: "9bb9602d-68d3-40c2-8561-805c2014e908"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.918633 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.918568    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb9602d-68d3-40c2-8561-805c2014e908-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9bb9602d-68d3-40c2-8561-805c2014e908" (UID: "9bb9602d-68d3-40c2-8561-805c2014e908"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:12.918690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.918629    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-kube-api-access-g4gkr" (OuterVolumeSpecName: "kube-api-access-g4gkr") pod "136ceb19-3aaf-4d94-beba-1dc83e8fcbab" (UID: "136ceb19-3aaf-4d94-beba-1dc83e8fcbab"). InnerVolumeSpecName "kube-api-access-g4gkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:12.918690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.918644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-dshm" (OuterVolumeSpecName: "dshm") pod "136ceb19-3aaf-4d94-beba-1dc83e8fcbab" (UID: "136ceb19-3aaf-4d94-beba-1dc83e8fcbab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:12.918690 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.918663    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "136ceb19-3aaf-4d94-beba-1dc83e8fcbab" (UID: "136ceb19-3aaf-4d94-beba-1dc83e8fcbab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:12.919143 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:12.919122    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb9602d-68d3-40c2-8561-805c2014e908-kube-api-access-qv2wr" (OuterVolumeSpecName: "kube-api-access-qv2wr") pod "9bb9602d-68d3-40c2-8561-805c2014e908" (UID: "9bb9602d-68d3-40c2-8561-805c2014e908"). InnerVolumeSpecName "kube-api-access-qv2wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:13.017902 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017864    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.017902 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017894    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv2wr\" (UniqueName: \"kubernetes.io/projected/9bb9602d-68d3-40c2-8561-805c2014e908-kube-api-access-qv2wr\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.017902 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017904    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.017902 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017912    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.018174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017922    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb9602d-68d3-40c2-8561-805c2014e908-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.018174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017931    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.018174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017938    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.018174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017946    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.018174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017953    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb9602d-68d3-40c2-8561-805c2014e908-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.018174 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.017960    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4gkr\" (UniqueName: \"kubernetes.io/projected/136ceb19-3aaf-4d94-beba-1dc83e8fcbab-kube-api-access-g4gkr\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:13.088953 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.088912    2560 generic.go:356] "Generic (PLEG): container finished" podID="9bb9602d-68d3-40c2-8561-805c2014e908" containerID="eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f" exitCode=0
Sep 10 14:56:13.089124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.088960    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"
Sep 10 14:56:13.089124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.089006    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2" event={"ID":"9bb9602d-68d3-40c2-8561-805c2014e908","Type":"ContainerDied","Data":"eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f"}
Sep 10 14:56:13.089124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.089051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2" event={"ID":"9bb9602d-68d3-40c2-8561-805c2014e908","Type":"ContainerDied","Data":"e665b54800833f6e9b0303b443c537ab4fd5ccb2a30d8824e4f0cdd7dbef51be"}
Sep 10 14:56:13.089124 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.089070    2560 scope.go:117] "RemoveContainer" containerID="eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f"
Sep 10 14:56:13.090427 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.090402    2560 generic.go:356] "Generic (PLEG): container finished" podID="585bbeb6-0249-433c-a2cd-7bcfb3823259" containerID="30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c" exitCode=0
Sep 10 14:56:13.090533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.090443    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"
Sep 10 14:56:13.090533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.090480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc" event={"ID":"585bbeb6-0249-433c-a2cd-7bcfb3823259","Type":"ContainerDied","Data":"30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c"}
Sep 10 14:56:13.090533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.090515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc" event={"ID":"585bbeb6-0249-433c-a2cd-7bcfb3823259","Type":"ContainerDied","Data":"b037e045adb78339230b740507748a5e9977a98b53d219e9d973690a49ffae80"}
Sep 10 14:56:13.092187 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.092163    2560 generic.go:356] "Generic (PLEG): container finished" podID="136ceb19-3aaf-4d94-beba-1dc83e8fcbab" containerID="316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc" exitCode=0
Sep 10 14:56:13.092351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.092198    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"
Sep 10 14:56:13.092351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.092195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc" event={"ID":"136ceb19-3aaf-4d94-beba-1dc83e8fcbab","Type":"ContainerDied","Data":"316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc"}
Sep 10 14:56:13.092351 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.092340    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc" event={"ID":"136ceb19-3aaf-4d94-beba-1dc83e8fcbab","Type":"ContainerDied","Data":"14e13b58a9353bc9a3bacbcfb9e508f9636bcce2049b9f0759be4fd47b02c24f"}
Sep 10 14:56:13.100025 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.100005    2560 scope.go:117] "RemoveContainer" containerID="eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f"
Sep 10 14:56:13.100343 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:13.100321    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f\": container with ID starting with eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f not found: ID does not exist" containerID="eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f"
Sep 10 14:56:13.100442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.100353    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f"} err="failed to get container status \"eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f\": rpc error: code = NotFound desc = could not find container \"eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f\": container with ID starting with eda71a5d620bc631fed9daff5097391e7a1efaea123821fdf1408f2b5cbb3c3f not found: ID does not exist"
Sep 10 14:56:13.100442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.100376    2560 scope.go:117] "RemoveContainer" containerID="30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c"
Sep 10 14:56:13.109533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.109516    2560 scope.go:117] "RemoveContainer" containerID="30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c"
Sep 10 14:56:13.109798 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:13.109783    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c\": container with ID starting with 30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c not found: ID does not exist" containerID="30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c"
Sep 10 14:56:13.109854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.109806    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c"} err="failed to get container status \"30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c\": rpc error: code = NotFound desc = could not find container \"30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c\": container with ID starting with 30068c0948556b75acefb71d3409e71550a241d7db0b7f39b7d390ed53a0f02c not found: ID does not exist"
Sep 10 14:56:13.109854 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.109822    2560 scope.go:117] "RemoveContainer" containerID="316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc"
Sep 10 14:56:13.117985 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.117958    2560 scope.go:117] "RemoveContainer" containerID="316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc"
Sep 10 14:56:13.118301 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:13.118261    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc\": container with ID starting with 316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc not found: ID does not exist" containerID="316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc"
Sep 10 14:56:13.118371 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.118299    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc"} err="failed to get container status \"316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc\": rpc error: code = NotFound desc = could not find container \"316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc\": container with ID starting with 316281519e9c07b80fdefcf46798afebd5205e7a47fa71c8ed0e45af4e99a4bc not found: ID does not exist"
Sep 10 14:56:13.155324 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.155293    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"]
Sep 10 14:56:13.161937 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.161909    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/distinct-2-w0-9b1e67-kserve-57c5fcfc7d-8hkkc"]
Sep 10 14:56:13.185132 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.185059    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"]
Sep 10 14:56:13.210206 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.210176    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/unconfigured-w0-9b1e67-kserve-75967d896b-gt7v2"]
Sep 10 14:56:13.259741 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.259662    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"]
Sep 10 14:56:13.264195 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.264172    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9b1e67/simulator-w0-9b1e67-kserve-57d9b948c-z5ntc"]
Sep 10 14:56:13.319013 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.318979    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="136ceb19-3aaf-4d94-beba-1dc83e8fcbab" path="/var/lib/kubelet/pods/136ceb19-3aaf-4d94-beba-1dc83e8fcbab/volumes"
Sep 10 14:56:13.319434 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.319420    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585bbeb6-0249-433c-a2cd-7bcfb3823259" path="/var/lib/kubelet/pods/585bbeb6-0249-433c-a2cd-7bcfb3823259/volumes"
Sep 10 14:56:13.319783 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:13.319770    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bb9602d-68d3-40c2-8561-805c2014e908" path="/var/lib/kubelet/pods/9bb9602d-68d3-40c2-8561-805c2014e908/volumes"
Sep 10 14:56:19.321244 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:19.321201    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:19.479676 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:19.479638    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:19.751397 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:19.751365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:22.411505 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:22.411475    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:22.711135 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:22.711094    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:27.252381 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:27.252344    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"]
Sep 10 14:56:27.431128 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:27.431093    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:36.544871 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:36.544835    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:56:38.370774 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.370739    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"]
Sep 10 14:56:38.371212 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.370930    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442" podUID="e3cad926-6a93-4025-be67-71ff4d11ca72" containerName="maas-api" containerID="cri-o://9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa" gracePeriod=30
Sep 10 14:56:38.551209 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.551176    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"]
Sep 10 14:56:38.551649 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.551439    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn" podUID="874b7593-06db-4f07-8a39-cbc00ad94e30" containerName="payload-pre-processing" containerID="cri-o://84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232" gracePeriod=30
Sep 10 14:56:38.611632 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.611601    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:38.683806 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.683766    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:56:38.735924 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.735889    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74zbf\" (UniqueName: \"kubernetes.io/projected/e3cad926-6a93-4025-be67-71ff4d11ca72-kube-api-access-74zbf\") pod \"e3cad926-6a93-4025-be67-71ff4d11ca72\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") "
Sep 10 14:56:38.736078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.735947    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cad926-6a93-4025-be67-71ff4d11ca72-openshift-service-ca\") pod \"e3cad926-6a93-4025-be67-71ff4d11ca72\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") "
Sep 10 14:56:38.736078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.735981    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-maas-api-tls\") pod \"e3cad926-6a93-4025-be67-71ff4d11ca72\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") "
Sep 10 14:56:38.736078 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.736039    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-metrics-certs\") pod \"e3cad926-6a93-4025-be67-71ff4d11ca72\" (UID: \"e3cad926-6a93-4025-be67-71ff4d11ca72\") "
Sep 10 14:56:38.736366 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.736338    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cad926-6a93-4025-be67-71ff4d11ca72-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3cad926-6a93-4025-be67-71ff4d11ca72" (UID: "e3cad926-6a93-4025-be67-71ff4d11ca72"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:56:38.738302 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.738251    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cad926-6a93-4025-be67-71ff4d11ca72-kube-api-access-74zbf" (OuterVolumeSpecName: "kube-api-access-74zbf") pod "e3cad926-6a93-4025-be67-71ff4d11ca72" (UID: "e3cad926-6a93-4025-be67-71ff4d11ca72"). InnerVolumeSpecName "kube-api-access-74zbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:38.738421 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.738359    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e3cad926-6a93-4025-be67-71ff4d11ca72" (UID: "e3cad926-6a93-4025-be67-71ff4d11ca72"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:38.738517 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.738492    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3cad926-6a93-4025-be67-71ff4d11ca72" (UID: "e3cad926-6a93-4025-be67-71ff4d11ca72"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:38.800607 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.800579    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:56:38.837311 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837259    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcd58\" (UniqueName: \"kubernetes.io/projected/874b7593-06db-4f07-8a39-cbc00ad94e30-kube-api-access-vcd58\") pod \"874b7593-06db-4f07-8a39-cbc00ad94e30\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") "
Sep 10 14:56:38.837511 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837341    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/874b7593-06db-4f07-8a39-cbc00ad94e30-plugins-config-volume\") pod \"874b7593-06db-4f07-8a39-cbc00ad94e30\" (UID: \"874b7593-06db-4f07-8a39-cbc00ad94e30\") "
Sep 10 14:56:38.837581 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837552    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cad926-6a93-4025-be67-71ff4d11ca72-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:38.837581 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837567    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:38.837693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837584    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cad926-6a93-4025-be67-71ff4d11ca72-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:38.837693 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837598    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74zbf\" (UniqueName: \"kubernetes.io/projected/e3cad926-6a93-4025-be67-71ff4d11ca72-kube-api-access-74zbf\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:38.837796 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.837707    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/874b7593-06db-4f07-8a39-cbc00ad94e30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "874b7593-06db-4f07-8a39-cbc00ad94e30" (UID: "874b7593-06db-4f07-8a39-cbc00ad94e30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:56:38.839849 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.839813    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874b7593-06db-4f07-8a39-cbc00ad94e30-kube-api-access-vcd58" (OuterVolumeSpecName: "kube-api-access-vcd58") pod "874b7593-06db-4f07-8a39-cbc00ad94e30" (UID: "874b7593-06db-4f07-8a39-cbc00ad94e30"). InnerVolumeSpecName "kube-api-access-vcd58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:38.902119 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.902025    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:38.938681 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.938648    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcd58\" (UniqueName: \"kubernetes.io/projected/874b7593-06db-4f07-8a39-cbc00ad94e30-kube-api-access-vcd58\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:38.938681 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:38.938678    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/874b7593-06db-4f07-8a39-cbc00ad94e30-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:39.065708 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.065675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:39.249411 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.249374    2560 generic.go:356] "Generic (PLEG): container finished" podID="874b7593-06db-4f07-8a39-cbc00ad94e30" containerID="84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232" exitCode=0
Sep 10 14:56:39.249578 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.249419    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"
Sep 10 14:56:39.249578 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.249452    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn" event={"ID":"874b7593-06db-4f07-8a39-cbc00ad94e30","Type":"ContainerDied","Data":"84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232"}
Sep 10 14:56:39.249578 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.249501    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn" event={"ID":"874b7593-06db-4f07-8a39-cbc00ad94e30","Type":"ContainerDied","Data":"76d3e1ff1acb8f730101d481f986f202fc844890f9fe1016416f3a1640aed1a9"}
Sep 10 14:56:39.249578 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.249523    2560 scope.go:117] "RemoveContainer" containerID="84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232"
Sep 10 14:56:39.250956 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.250930    2560 generic.go:356] "Generic (PLEG): container finished" podID="e3cad926-6a93-4025-be67-71ff4d11ca72" containerID="9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa" exitCode=0
Sep 10 14:56:39.251067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.250967    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"
Sep 10 14:56:39.251067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.251008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442" event={"ID":"e3cad926-6a93-4025-be67-71ff4d11ca72","Type":"ContainerDied","Data":"9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa"}
Sep 10 14:56:39.251067 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.251055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442" event={"ID":"e3cad926-6a93-4025-be67-71ff4d11ca72","Type":"ContainerDied","Data":"82b1f776b3ab266dc51547dadb6f56a73d0100ca863bb75a8aa077f97706eb88"}
Sep 10 14:56:39.260007 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.259989    2560 scope.go:117] "RemoveContainer" containerID="84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232"
Sep 10 14:56:39.260322 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:39.260302    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232\": container with ID starting with 84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232 not found: ID does not exist" containerID="84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232"
Sep 10 14:56:39.260399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.260330    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232"} err="failed to get container status \"84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232\": rpc error: code = NotFound desc = could not find container \"84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232\": container with ID starting with 84896d5f71577f31bfe60b6f6d9bc205b1811001fac60468f21cf8a4d9b34232 not found: ID does not exist"
Sep 10 14:56:39.260399 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.260345    2560 scope.go:117] "RemoveContainer" containerID="9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa"
Sep 10 14:56:39.268980 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.268963    2560 scope.go:117] "RemoveContainer" containerID="9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa"
Sep 10 14:56:39.269263 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:39.269243    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa\": container with ID starting with 9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa not found: ID does not exist" containerID="9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa"
Sep 10 14:56:39.269343 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.269270    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa"} err="failed to get container status \"9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa\": rpc error: code = NotFound desc = could not find container \"9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa\": container with ID starting with 9470fa856537f8b8483d3ccd7b9f2456e06178cdfe84115a1c66d62dec0299fa not found: ID does not exist"
Sep 10 14:56:39.298133 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.298106    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"]
Sep 10 14:56:39.306116 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.306085    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5b0323-cc8948d7f-h6442"]
Sep 10 14:56:39.322509 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.322453    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3cad926-6a93-4025-be67-71ff4d11ca72" path="/var/lib/kubelet/pods/e3cad926-6a93-4025-be67-71ff4d11ca72/volumes"
Sep 10 14:56:39.323238 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.323216    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"]
Sep 10 14:56:39.333861 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.333836    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5b0323-599dd9466-kndnn"]
Sep 10 14:56:39.446876 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.446844    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:39.565098 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:39.565009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:40.500639 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:40.500605    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:41.319516 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:41.319483    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="874b7593-06db-4f07-8a39-cbc00ad94e30" path="/var/lib/kubelet/pods/874b7593-06db-4f07-8a39-cbc00ad94e30/volumes"
Sep 10 14:56:42.271823 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.271785    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8" podUID="4ebd05e7-18d6-4005-ac8f-77313ccaaa25" containerName="main" containerID="cri-o://f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa" gracePeriod=45
Sep 10 14:56:42.534915 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.534855    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:56:42.570015 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.569980    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-model-cache\") pod \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") "
Sep 10 14:56:42.570207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570038    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-dshm\") pod \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") "
Sep 10 14:56:42.570207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570069    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcbw7\" (UniqueName: \"kubernetes.io/projected/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-kube-api-access-rcbw7\") pod \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") "
Sep 10 14:56:42.570207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570096    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-home\") pod \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") "
Sep 10 14:56:42.570207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570123    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tmp-dir\") pod \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") "
Sep 10 14:56:42.570207 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570201    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tls-certs\") pod \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\" (UID: \"4ebd05e7-18d6-4005-ac8f-77313ccaaa25\") "
Sep 10 14:56:42.570521 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570274    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ebd05e7-18d6-4005-ac8f-77313ccaaa25" (UID: "4ebd05e7-18d6-4005-ac8f-77313ccaaa25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:42.570521 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570407    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-home" (OuterVolumeSpecName: "home") pod "4ebd05e7-18d6-4005-ac8f-77313ccaaa25" (UID: "4ebd05e7-18d6-4005-ac8f-77313ccaaa25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:42.570521 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570493    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ebd05e7-18d6-4005-ac8f-77313ccaaa25" (UID: "4ebd05e7-18d6-4005-ac8f-77313ccaaa25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:42.570681 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570542    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:42.570681 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.570561    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:42.572463 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.572422    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-dshm" (OuterVolumeSpecName: "dshm") pod "4ebd05e7-18d6-4005-ac8f-77313ccaaa25" (UID: "4ebd05e7-18d6-4005-ac8f-77313ccaaa25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:56:42.572620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.572587    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ebd05e7-18d6-4005-ac8f-77313ccaaa25" (UID: "4ebd05e7-18d6-4005-ac8f-77313ccaaa25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:42.572781 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.572756    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-kube-api-access-rcbw7" (OuterVolumeSpecName: "kube-api-access-rcbw7") pod "4ebd05e7-18d6-4005-ac8f-77313ccaaa25" (UID: "4ebd05e7-18d6-4005-ac8f-77313ccaaa25"). InnerVolumeSpecName "kube-api-access-rcbw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:42.671442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.671408    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:42.671442 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.671440    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcbw7\" (UniqueName: \"kubernetes.io/projected/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-kube-api-access-rcbw7\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:42.671639 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.671451    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:42.671639 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:42.671460    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ebd05e7-18d6-4005-ac8f-77313ccaaa25-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:56:43.280114 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.280076    2560 generic.go:356] "Generic (PLEG): container finished" podID="4ebd05e7-18d6-4005-ac8f-77313ccaaa25" containerID="f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa" exitCode=0
Sep 10 14:56:43.280555 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.280121    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"
Sep 10 14:56:43.280555 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.280156    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8" event={"ID":"4ebd05e7-18d6-4005-ac8f-77313ccaaa25","Type":"ContainerDied","Data":"f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa"}
Sep 10 14:56:43.280555 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.280202    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8" event={"ID":"4ebd05e7-18d6-4005-ac8f-77313ccaaa25","Type":"ContainerDied","Data":"2f5ea2114fda6617428adc9fbee3214c410181dd44ece91323940959c07413f4"}
Sep 10 14:56:43.280555 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.280221    2560 scope.go:117] "RemoveContainer" containerID="f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa"
Sep 10 14:56:43.289620 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.289600    2560 scope.go:117] "RemoveContainer" containerID="f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa"
Sep 10 14:56:43.289900 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:56:43.289879    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa\": container with ID starting with f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa not found: ID does not exist" containerID="f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa"
Sep 10 14:56:43.289972 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.289914    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa"} err="failed to get container status \"f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa\": rpc error: code = NotFound desc = could not find container \"f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa\": container with ID starting with f20a1cfc9d84db4787f3e11074cb8c62dcc8a7e5d2872f558c33d920804ba1aa not found: ID does not exist"
Sep 10 14:56:43.319112 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.319084    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"]
Sep 10 14:56:43.325091 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:43.325066    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-5b0323/premium-w4-5b0323-kserve-6449d484b7-9wbs8"]
Sep 10 14:56:45.318460 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:45.318416    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ebd05e7-18d6-4005-ac8f-77313ccaaa25" path="/var/lib/kubelet/pods/4ebd05e7-18d6-4005-ac8f-77313ccaaa25/volumes"
Sep 10 14:56:50.151010 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:50.150979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:50.337186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:50.337144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:50.598328 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:50.598261    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:51.704533 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:51.704496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:57.528104 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:57.528073    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:56:58.859722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:56:58.859675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:03.332837 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:03.332808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:03.802287 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:03.802239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:03.970332 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:03.970264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:08.821796 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:08.821752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:12.620413 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:12.620381    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:57:16.027500 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:16.027456    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:26.476694 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.476662    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"]
Sep 10 14:57:26.477041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477029    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" containerName="maas-api"
Sep 10 14:57:26.477041 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477039    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" containerName="maas-api"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477049    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="136ceb19-3aaf-4d94-beba-1dc83e8fcbab" containerName="main"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477055    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="136ceb19-3aaf-4d94-beba-1dc83e8fcbab" containerName="main"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477061    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2be7141-d1bb-4a32-9218-f36497842ad5" containerName="payload-processing"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477067    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2be7141-d1bb-4a32-9218-f36497842ad5" containerName="payload-processing"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477076    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874b7593-06db-4f07-8a39-cbc00ad94e30" containerName="payload-pre-processing"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477081    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="874b7593-06db-4f07-8a39-cbc00ad94e30" containerName="payload-pre-processing"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477090    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e90498b0-6417-4d3b-bd43-f5ed29eca51b" containerName="main"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477095    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e90498b0-6417-4d3b-bd43-f5ed29eca51b" containerName="main"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477106    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585bbeb6-0249-433c-a2cd-7bcfb3823259" containerName="main"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477111    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="585bbeb6-0249-433c-a2cd-7bcfb3823259" containerName="main"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477116    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2116c427-8cb4-4383-95ce-4386943e5546" containerName="payload-processing"
Sep 10 14:57:26.477118 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477121    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2116c427-8cb4-4383-95ce-4386943e5546" containerName="payload-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477132    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c76b901-4e73-4e90-8b9b-56cdb0263ba9" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477137    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c76b901-4e73-4e90-8b9b-56cdb0263ba9" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477143    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb9602d-68d3-40c2-8561-805c2014e908" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477147    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb9602d-68d3-40c2-8561-805c2014e908" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477155    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cad926-6a93-4025-be67-71ff4d11ca72" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477161    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cad926-6a93-4025-be67-71ff4d11ca72" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477169    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ebd05e7-18d6-4005-ac8f-77313ccaaa25" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477174    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ebd05e7-18d6-4005-ac8f-77313ccaaa25" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477181    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41064590-4693-429b-9892-eefc37ce1a68" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477186    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41064590-4693-429b-9892-eefc37ce1a68" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477192    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477196    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477202    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ecb0487-f599-4a1d-9328-93f92aa7030e" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477206    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ecb0487-f599-4a1d-9328-93f92aa7030e" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477212    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd58eb8-75c8-462f-bf2e-58919503e4a4" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477216    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd58eb8-75c8-462f-bf2e-58919503e4a4" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477273    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="170694a2-f69b-4ef7-9cdd-0d6ad95ddc4a" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477297    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ebd05e7-18d6-4005-ac8f-77313ccaaa25" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477302    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="585bbeb6-0249-433c-a2cd-7bcfb3823259" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477308    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec1f670e-3d8f-415c-b1ce-51e48eb96c8b" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477314    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c76b901-4e73-4e90-8b9b-56cdb0263ba9" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477322    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb9602d-68d3-40c2-8561-805c2014e908" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477327    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd58eb8-75c8-462f-bf2e-58919503e4a4" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477333    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e90498b0-6417-4d3b-bd43-f5ed29eca51b" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477338    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ecb0487-f599-4a1d-9328-93f92aa7030e" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477344    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="136ceb19-3aaf-4d94-beba-1dc83e8fcbab" containerName="main"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477350    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cad926-6a93-4025-be67-71ff4d11ca72" containerName="maas-api"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477355    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="874b7593-06db-4f07-8a39-cbc00ad94e30" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477360    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="41064590-4693-429b-9892-eefc37ce1a68" containerName="payload-pre-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477365    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2116c427-8cb4-4383-95ce-4386943e5546" containerName="payload-processing"
Sep 10 14:57:26.477493 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.477370    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2be7141-d1bb-4a32-9218-f36497842ad5" containerName="payload-processing"
Sep 10 14:57:26.479948 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.479932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.486523 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.486489    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"kube-root-ca.crt\""
Sep 10 14:57:26.486683 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.486620    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"openshift-service-ca.crt\""
Sep 10 14:57:26.487437 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.487419    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"sub-test-model-12b6f6-kserve-self-signed-certs\""
Sep 10 14:57:26.504180 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.504150    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"default-dockercfg-lrmnz\""
Sep 10 14:57:26.553311 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.553267    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"]
Sep 10 14:57:26.581563 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.581531    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-dshm\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.581563 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.581562    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlk6c\" (UniqueName: \"kubernetes.io/projected/5529f999-e3c6-4686-ad85-317684d7915b-kube-api-access-vlk6c\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.581907 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.581589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-home\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.581907 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.581668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-tmp-dir\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.581907 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.581825    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529f999-e3c6-4686-ad85-317684d7915b-tls-certs\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.582097 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.581961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-model-cache\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.605957 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.605924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:26.682668 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.682630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-home\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.682838 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.682676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-tmp-dir\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.682838 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.682737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529f999-e3c6-4686-ad85-317684d7915b-tls-certs\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.682838 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.682763    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-model-cache\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.682838 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.682792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-dshm\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.682838 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.682815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlk6c\" (UniqueName: \"kubernetes.io/projected/5529f999-e3c6-4686-ad85-317684d7915b-kube-api-access-vlk6c\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.683137 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.683113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-home\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.683211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.683131    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-tmp-dir\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.683211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.683156    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-model-cache\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.685074 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.685042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-dshm\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.685376 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.685360    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529f999-e3c6-4686-ad85-317684d7915b-tls-certs\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.702763 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.702730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlk6c\" (UniqueName: \"kubernetes.io/projected/5529f999-e3c6-4686-ad85-317684d7915b-kube-api-access-vlk6c\") pod \"sub-test-model-12b6f6-kserve-7767667977-wfslx\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:26.789532 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:26.789448    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:27.073572 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:27.073542    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"]
Sep 10 14:57:27.075601 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:57:27.075564    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5529f999_e3c6_4686_ad85_317684d7915b.slice/crio-903a2f128dc504d8bf34ffdd83e3d5d88eb9f866de2f40e8968956382ea926e4 WatchSource:0}: Error finding container 903a2f128dc504d8bf34ffdd83e3d5d88eb9f866de2f40e8968956382ea926e4: Status 404 returned error can't find the container with id 903a2f128dc504d8bf34ffdd83e3d5d88eb9f866de2f40e8968956382ea926e4
Sep 10 14:57:27.551448 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:27.551403    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx" event={"ID":"5529f999-e3c6-4686-ad85-317684d7915b","Type":"ContainerStarted","Data":"d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518"}
Sep 10 14:57:27.551868 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:27.551462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx" event={"ID":"5529f999-e3c6-4686-ad85-317684d7915b","Type":"ContainerStarted","Data":"903a2f128dc504d8bf34ffdd83e3d5d88eb9f866de2f40e8968956382ea926e4"}
Sep 10 14:57:29.654028 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:29.653965    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx" podStartSLOduration=3.653946096 podStartE2EDuration="3.653946096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:57:27.590216812 +0000 UTC m=+1776.946029730" watchObservedRunningTime="2026-09-10 14:57:29.653946096 +0000 UTC m=+1779.009759035"
Sep 10 14:57:29.654675 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:29.654658    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:57:33.215963 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:33.215932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:36.790364 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:36.790306    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:36.790861 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:36.790393    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:36.803428 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:36.803393    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:37.627543 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:37.627513    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:57:38.382449 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:38.382413    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:52.810032 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:52.809994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:57:58.673318 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:57:58.673273    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:58:02.257718 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:02.257684    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:58:17.497590 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:17.497555    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:58:21.310644 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:21.310607    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:58:31.512033 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:31.511998    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:58:39.579435 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:39.579398    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"]
Sep 10 14:58:39.645341 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:39.645304    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:58:40.525222 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:40.525187    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:58:43.038853 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:43.038814    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:58:54.599189 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.599138    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx" podUID="5529f999-e3c6-4686-ad85-317684d7915b" containerName="main" containerID="cri-o://d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518" gracePeriod=45
Sep 10 14:58:54.856056 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.855998    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:58:54.902749 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.902716    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-dshm\") pod \"5529f999-e3c6-4686-ad85-317684d7915b\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") "
Sep 10 14:58:54.902923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.902762    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlk6c\" (UniqueName: \"kubernetes.io/projected/5529f999-e3c6-4686-ad85-317684d7915b-kube-api-access-vlk6c\") pod \"5529f999-e3c6-4686-ad85-317684d7915b\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") "
Sep 10 14:58:54.902923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.902784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-model-cache\") pod \"5529f999-e3c6-4686-ad85-317684d7915b\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") "
Sep 10 14:58:54.902923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.902839    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-home\") pod \"5529f999-e3c6-4686-ad85-317684d7915b\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") "
Sep 10 14:58:54.902923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.902873    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529f999-e3c6-4686-ad85-317684d7915b-tls-certs\") pod \"5529f999-e3c6-4686-ad85-317684d7915b\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") "
Sep 10 14:58:54.902923 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.902905    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-tmp-dir\") pod \"5529f999-e3c6-4686-ad85-317684d7915b\" (UID: \"5529f999-e3c6-4686-ad85-317684d7915b\") "
Sep 10 14:58:54.903211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.903089    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-model-cache" (OuterVolumeSpecName: "model-cache") pod "5529f999-e3c6-4686-ad85-317684d7915b" (UID: "5529f999-e3c6-4686-ad85-317684d7915b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:54.903211 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.903116    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-home" (OuterVolumeSpecName: "home") pod "5529f999-e3c6-4686-ad85-317684d7915b" (UID: "5529f999-e3c6-4686-ad85-317684d7915b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:54.903356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.903267    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:54.903356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.903305    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:54.903356 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.903316    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5529f999-e3c6-4686-ad85-317684d7915b" (UID: "5529f999-e3c6-4686-ad85-317684d7915b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:54.904963 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.904942    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-dshm" (OuterVolumeSpecName: "dshm") pod "5529f999-e3c6-4686-ad85-317684d7915b" (UID: "5529f999-e3c6-4686-ad85-317684d7915b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:54.905309 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.905256    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5529f999-e3c6-4686-ad85-317684d7915b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5529f999-e3c6-4686-ad85-317684d7915b" (UID: "5529f999-e3c6-4686-ad85-317684d7915b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:58:54.905398 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:54.905308    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5529f999-e3c6-4686-ad85-317684d7915b-kube-api-access-vlk6c" (OuterVolumeSpecName: "kube-api-access-vlk6c") pod "5529f999-e3c6-4686-ad85-317684d7915b" (UID: "5529f999-e3c6-4686-ad85-317684d7915b"). InnerVolumeSpecName "kube-api-access-vlk6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:58:55.003872 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.003837    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529f999-e3c6-4686-ad85-317684d7915b-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:55.003872 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.003863    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:55.003872 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.003871    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529f999-e3c6-4686-ad85-317684d7915b-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:55.003872 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.003880    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlk6c\" (UniqueName: \"kubernetes.io/projected/5529f999-e3c6-4686-ad85-317684d7915b-kube-api-access-vlk6c\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:55.053102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.053064    2560 generic.go:356] "Generic (PLEG): container finished" podID="5529f999-e3c6-4686-ad85-317684d7915b" containerID="d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518" exitCode=0
Sep 10 14:58:55.053102 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.053100    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"
Sep 10 14:58:55.053314 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.053142    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx" event={"ID":"5529f999-e3c6-4686-ad85-317684d7915b","Type":"ContainerDied","Data":"d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518"}
Sep 10 14:58:55.053314 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.053189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx" event={"ID":"5529f999-e3c6-4686-ad85-317684d7915b","Type":"ContainerDied","Data":"903a2f128dc504d8bf34ffdd83e3d5d88eb9f866de2f40e8968956382ea926e4"}
Sep 10 14:58:55.053314 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.053207    2560 scope.go:117] "RemoveContainer" containerID="d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518"
Sep 10 14:58:55.063370 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.063352    2560 scope.go:117] "RemoveContainer" containerID="d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518"
Sep 10 14:58:55.063652 ip-10-0-132-253 kubenswrapper[2560]: E0910 14:58:55.063629    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518\": container with ID starting with d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518 not found: ID does not exist" containerID="d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518"
Sep 10 14:58:55.063698 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.063659    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518"} err="failed to get container status \"d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518\": rpc error: code = NotFound desc = could not find container \"d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518\": container with ID starting with d309cf0bc3b51a4e8e8dd3cb9bbcc975b1a1d63dfaec6d50db2f484f4a2ff518 not found: ID does not exist"
Sep 10 14:58:55.078052 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.078028    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"]
Sep 10 14:58:55.080699 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.080678    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/sub-test-model-12b6f6-kserve-7767667977-wfslx"]
Sep 10 14:58:55.321954 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:55.321916    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5529f999-e3c6-4686-ad85-317684d7915b" path="/var/lib/kubelet/pods/5529f999-e3c6-4686-ad85-317684d7915b/volumes"
Sep 10 14:58:58.539438 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:58:58.539402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:03.954032 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.953999    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"]
Sep 10 14:59:03.954497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.954419    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5529f999-e3c6-4686-ad85-317684d7915b" containerName="main"
Sep 10 14:59:03.954497 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.954432    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5529f999-e3c6-4686-ad85-317684d7915b" containerName="main"
Sep 10 14:59:03.954569 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.954508    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5529f999-e3c6-4686-ad85-317684d7915b" containerName="main"
Sep 10 14:59:03.958846 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.958823    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.961842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.961765    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"rate-test-model-12b6f6-kserve-self-signed-certs\""
Sep 10 14:59:03.961842 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.961766    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"kube-root-ca.crt\""
Sep 10 14:59:03.962023 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.961767    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"default-dockercfg-lrmnz\""
Sep 10 14:59:03.962023 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.961776    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-12b6f6\"/\"openshift-service-ca.crt\""
Sep 10 14:59:03.963621 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.963597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d08ab1d-990b-4936-a48b-ecdfa042e960-tls-certs\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.963722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.963641    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-model-cache\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.963722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.963676    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-tmp-dir\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.963722 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.963701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-dshm\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.963852 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.963720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcc2n\" (UniqueName: \"kubernetes.io/projected/9d08ab1d-990b-4936-a48b-ecdfa042e960-kube-api-access-fcc2n\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.963852 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.963754    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-home\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:03.971844 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.971817    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"]
Sep 10 14:59:03.979055 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:03.979007    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:04.064334 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064269    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-home\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d08ab1d-990b-4936-a48b-ecdfa042e960-tls-certs\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-model-cache\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064417    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-tmp-dir\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-dshm\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064507 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064467    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcc2n\" (UniqueName: \"kubernetes.io/projected/9d08ab1d-990b-4936-a48b-ecdfa042e960-kube-api-access-fcc2n\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064717    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-home\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064786 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064768    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-tmp-dir\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.064894 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.064833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-model-cache\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.066661 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.066641    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-dshm\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.066952 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.066932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d08ab1d-990b-4936-a48b-ecdfa042e960-tls-certs\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.077186 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.077154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcc2n\" (UniqueName: \"kubernetes.io/projected/9d08ab1d-990b-4936-a48b-ecdfa042e960-kube-api-access-fcc2n\") pod \"rate-test-model-12b6f6-kserve-6f8868c559-2jcpj\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") " pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.268514 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.268418    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:04.408811 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.408775    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:04.515197 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.515157    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:04.545617 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:04.545589    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"]
Sep 10 14:59:04.547138 ip-10-0-132-253 kubenswrapper[2560]: W0910 14:59:04.547114    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d08ab1d_990b_4936_a48b_ecdfa042e960.slice/crio-bd8b14b9a010cf29a8b105a7a3442430509bbb32606be92f3a3c6c7c4fdfc91e WatchSource:0}: Error finding container bd8b14b9a010cf29a8b105a7a3442430509bbb32606be92f3a3c6c7c4fdfc91e: Status 404 returned error can't find the container with id bd8b14b9a010cf29a8b105a7a3442430509bbb32606be92f3a3c6c7c4fdfc91e
Sep 10 14:59:05.110467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:05.110429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj" event={"ID":"9d08ab1d-990b-4936-a48b-ecdfa042e960","Type":"ContainerStarted","Data":"0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551"}
Sep 10 14:59:05.110467 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:05.110470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj" event={"ID":"9d08ab1d-990b-4936-a48b-ecdfa042e960","Type":"ContainerStarted","Data":"bd8b14b9a010cf29a8b105a7a3442430509bbb32606be92f3a3c6c7c4fdfc91e"}
Sep 10 14:59:05.144092 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:05.144039    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj" podStartSLOduration=2.144019618 podStartE2EDuration="2.144019618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:05.142306854 +0000 UTC m=+1874.498119781" watchObservedRunningTime="2026-09-10 14:59:05.144019618 +0000 UTC m=+1874.499832526"
Sep 10 14:59:06.915906 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:06.915864    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:59:14.269120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:14.269034    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:14.269120 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:14.269090    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:14.281728 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:14.281691    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:15.186020 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:15.185987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 14:59:25.782181 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:25.782145    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:29.753144 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:29.753104    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 14:59:54.126425 ip-10-0-132-253 kubenswrapper[2560]: I0910 14:59:54.126384    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:00:03.698342 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:03.698307    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:00:10.439683 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:10.439646    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"]
Sep 10 15:00:11.048478 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:11.048448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:11.356408 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:11.356327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:11.807456 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:11.807421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:12.218982 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:12.218951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:12.645517 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:12.645435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:14.833983 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:14.833947    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:00:16.381858 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:16.381800    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:16.516054 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:16.516025    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:17.747459 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:17.747430    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:18.028456 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:18.028356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:18.186309 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:18.186264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:18.864290 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:18.864244    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:19.002644 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.002609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:19.071693 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.071661    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"]
Sep 10 15:00:19.075816 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.075796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.079543 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.079517    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-ae0f24-dockercfg-wnwrd\""
Sep 10 15:00:19.079543 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.079531    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-ae0f24\""
Sep 10 15:00:19.102818 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.102787    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"]
Sep 10 15:00:19.105226 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.105201    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"]
Sep 10 15:00:19.108541 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.108520    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.132416 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.132394    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"]
Sep 10 15:00:19.140429 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.140404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e3b266b-b308-404a-987c-ae39d3480513-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.140541 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.140471    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcsbf\" (UniqueName: \"kubernetes.io/projected/7e3b266b-b308-404a-987c-ae39d3480513-kube-api-access-gcsbf\") pod \"payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.241860 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.241824    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5038e7c1-60f9-47da-8e50-aace901afcf7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.242046 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.241872    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e3b266b-b308-404a-987c-ae39d3480513-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.242046 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.241935    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vnsc\" (UniqueName: \"kubernetes.io/projected/5038e7c1-60f9-47da-8e50-aace901afcf7-kube-api-access-4vnsc\") pod \"payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.242046 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.241976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcsbf\" (UniqueName: \"kubernetes.io/projected/7e3b266b-b308-404a-987c-ae39d3480513-kube-api-access-gcsbf\") pod \"payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.242613 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.242596    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e3b266b-b308-404a-987c-ae39d3480513-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.256242 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.256217    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcsbf\" (UniqueName: \"kubernetes.io/projected/7e3b266b-b308-404a-987c-ae39d3480513-kube-api-access-gcsbf\") pod \"payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.342531 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.342487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5038e7c1-60f9-47da-8e50-aace901afcf7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.342726 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.342567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vnsc\" (UniqueName: \"kubernetes.io/projected/5038e7c1-60f9-47da-8e50-aace901afcf7-kube-api-access-4vnsc\") pod \"payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.343169 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.343140    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5038e7c1-60f9-47da-8e50-aace901afcf7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.353735 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.353709    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vnsc\" (UniqueName: \"kubernetes.io/projected/5038e7c1-60f9-47da-8e50-aace901afcf7-kube-api-access-4vnsc\") pod \"payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.386064 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.385994    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:19.418016 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.417979    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:19.667784 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.667759    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"]
Sep 10 15:00:19.669209 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:00:19.669187    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3b266b_b308_404a_987c_ae39d3480513.slice/crio-93f253eada8adcebbd1c5a0cb40dd76979ead46407fe66c3166d83517b5cc967 WatchSource:0}: Error finding container 93f253eada8adcebbd1c5a0cb40dd76979ead46407fe66c3166d83517b5cc967: Status 404 returned error can't find the container with id 93f253eada8adcebbd1c5a0cb40dd76979ead46407fe66c3166d83517b5cc967
Sep 10 15:00:19.711754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:19.711734    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"]
Sep 10 15:00:19.713716 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:00:19.713695    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5038e7c1_60f9_47da_8e50_aace901afcf7.slice/crio-b2129c51cfa3a13a9882bf3e49146964a70a9472409867c69e9c85eaa45bcbe2 WatchSource:0}: Error finding container b2129c51cfa3a13a9882bf3e49146964a70a9472409867c69e9c85eaa45bcbe2: Status 404 returned error can't find the container with id b2129c51cfa3a13a9882bf3e49146964a70a9472409867c69e9c85eaa45bcbe2
Sep 10 15:00:20.538149 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.538106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2" event={"ID":"5038e7c1-60f9-47da-8e50-aace901afcf7","Type":"ContainerStarted","Data":"b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c"}
Sep 10 15:00:20.538598 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.538169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2" event={"ID":"5038e7c1-60f9-47da-8e50-aace901afcf7","Type":"ContainerStarted","Data":"b2129c51cfa3a13a9882bf3e49146964a70a9472409867c69e9c85eaa45bcbe2"}
Sep 10 15:00:20.538598 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.538224    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:20.539786 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.539756    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm" event={"ID":"7e3b266b-b308-404a-987c-ae39d3480513","Type":"ContainerStarted","Data":"e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75"}
Sep 10 15:00:20.539914 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.539803    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm" event={"ID":"7e3b266b-b308-404a-987c-ae39d3480513","Type":"ContainerStarted","Data":"93f253eada8adcebbd1c5a0cb40dd76979ead46407fe66c3166d83517b5cc967"}
Sep 10 15:00:20.539914 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.539832    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:20.567640 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.567596    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2" podStartSLOduration=1.5675818449999999 podStartE2EDuration="1.567581845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:20.563741819 +0000 UTC m=+1949.919554736" watchObservedRunningTime="2026-09-10 15:00:20.567581845 +0000 UTC m=+1949.923394751"
Sep 10 15:00:20.596429 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:20.596368    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm" podStartSLOduration=1.5963493789999998 podStartE2EDuration="1.596349379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:20.595545616 +0000 UTC m=+1949.951358549" watchObservedRunningTime="2026-09-10 15:00:20.596349379 +0000 UTC m=+1949.952162286"
Sep 10 15:00:25.458651 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.458605    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj" podUID="9d08ab1d-990b-4936-a48b-ecdfa042e960" containerName="main" containerID="cri-o://0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551" gracePeriod=45
Sep 10 15:00:25.716376 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.716314    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 15:00:25.804576 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804529    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-home\") pod \"9d08ab1d-990b-4936-a48b-ecdfa042e960\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") "
Sep 10 15:00:25.804754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804599    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-dshm\") pod \"9d08ab1d-990b-4936-a48b-ecdfa042e960\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") "
Sep 10 15:00:25.804754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804630    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d08ab1d-990b-4936-a48b-ecdfa042e960-tls-certs\") pod \"9d08ab1d-990b-4936-a48b-ecdfa042e960\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") "
Sep 10 15:00:25.804754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804657    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-tmp-dir\") pod \"9d08ab1d-990b-4936-a48b-ecdfa042e960\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") "
Sep 10 15:00:25.804754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804684    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-model-cache\") pod \"9d08ab1d-990b-4936-a48b-ecdfa042e960\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") "
Sep 10 15:00:25.804754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804733    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcc2n\" (UniqueName: \"kubernetes.io/projected/9d08ab1d-990b-4936-a48b-ecdfa042e960-kube-api-access-fcc2n\") pod \"9d08ab1d-990b-4936-a48b-ecdfa042e960\" (UID: \"9d08ab1d-990b-4936-a48b-ecdfa042e960\") "
Sep 10 15:00:25.804997 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804860    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-home" (OuterVolumeSpecName: "home") pod "9d08ab1d-990b-4936-a48b-ecdfa042e960" (UID: "9d08ab1d-990b-4936-a48b-ecdfa042e960"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:00:25.804997 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.804967    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d08ab1d-990b-4936-a48b-ecdfa042e960" (UID: "9d08ab1d-990b-4936-a48b-ecdfa042e960"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:00:25.805113 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.805003    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d08ab1d-990b-4936-a48b-ecdfa042e960" (UID: "9d08ab1d-990b-4936-a48b-ecdfa042e960"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:00:25.805193 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.805168    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:00:25.805252 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.805201    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:00:25.805252 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.805216    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:00:25.806825 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.806798    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-dshm" (OuterVolumeSpecName: "dshm") pod "9d08ab1d-990b-4936-a48b-ecdfa042e960" (UID: "9d08ab1d-990b-4936-a48b-ecdfa042e960"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:00:25.806936 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.806849    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d08ab1d-990b-4936-a48b-ecdfa042e960-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d08ab1d-990b-4936-a48b-ecdfa042e960" (UID: "9d08ab1d-990b-4936-a48b-ecdfa042e960"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:00:25.806936 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.806894    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d08ab1d-990b-4936-a48b-ecdfa042e960-kube-api-access-fcc2n" (OuterVolumeSpecName: "kube-api-access-fcc2n") pod "9d08ab1d-990b-4936-a48b-ecdfa042e960" (UID: "9d08ab1d-990b-4936-a48b-ecdfa042e960"). InnerVolumeSpecName "kube-api-access-fcc2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:00:25.906219 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.906186    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d08ab1d-990b-4936-a48b-ecdfa042e960-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:00:25.906219 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.906212    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d08ab1d-990b-4936-a48b-ecdfa042e960-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:00:25.906438 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:25.906229    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcc2n\" (UniqueName: \"kubernetes.io/projected/9d08ab1d-990b-4936-a48b-ecdfa042e960-kube-api-access-fcc2n\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:00:26.573176 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.573137    2560 generic.go:356] "Generic (PLEG): container finished" podID="9d08ab1d-990b-4936-a48b-ecdfa042e960" containerID="0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551" exitCode=0
Sep 10 15:00:26.573685 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.573192    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"
Sep 10 15:00:26.573685 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.573218    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj" event={"ID":"9d08ab1d-990b-4936-a48b-ecdfa042e960","Type":"ContainerDied","Data":"0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551"}
Sep 10 15:00:26.573685 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.573300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj" event={"ID":"9d08ab1d-990b-4936-a48b-ecdfa042e960","Type":"ContainerDied","Data":"bd8b14b9a010cf29a8b105a7a3442430509bbb32606be92f3a3c6c7c4fdfc91e"}
Sep 10 15:00:26.573685 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.573329    2560 scope.go:117] "RemoveContainer" containerID="0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551"
Sep 10 15:00:26.582593 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.582578    2560 scope.go:117] "RemoveContainer" containerID="0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551"
Sep 10 15:00:26.582865 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:00:26.582848    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551\": container with ID starting with 0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551 not found: ID does not exist" containerID="0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551"
Sep 10 15:00:26.582923 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.582871    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551"} err="failed to get container status \"0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551\": rpc error: code = NotFound desc = could not find container \"0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551\": container with ID starting with 0ca49cf1de87783ff374a4b45698889ed90c4df390f404faf29a21f548ee4551 not found: ID does not exist"
Sep 10 15:00:26.620765 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.620742    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"]
Sep 10 15:00:26.630259 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:26.630232    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-12b6f6/rate-test-model-12b6f6-kserve-6f8868c559-2jcpj"]
Sep 10 15:00:27.322739 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:27.322709    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d08ab1d-990b-4936-a48b-ecdfa042e960" path="/var/lib/kubelet/pods/9d08ab1d-990b-4936-a48b-ecdfa042e960/volumes"
Sep 10 15:00:30.612264 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:30.612231    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:00:31.547442 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:31.547412    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:00:31.547612 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:31.547482    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:00:46.542573 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:46.542472    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:47.257355 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:47.257319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:47.643708 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:47.643626    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:47.973092 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:47.973047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:50.807796 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:50.807763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:50.985053 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:50.985023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:52.277513 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:52.277470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:52.637735 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:52.637652    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:53.774243 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:53.774198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:54.055401 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.055313    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:54.273252 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.273222    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"]
Sep 10 15:00:54.273668 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.273654    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d08ab1d-990b-4936-a48b-ecdfa042e960" containerName="main"
Sep 10 15:00:54.273718 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.273671    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d08ab1d-990b-4936-a48b-ecdfa042e960" containerName="main"
Sep 10 15:00:54.273756 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.273737    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d08ab1d-990b-4936-a48b-ecdfa042e960" containerName="main"
Sep 10 15:00:54.278188 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.278171    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.282359 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.282337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-b8f4d6-dockercfg-vjlnq\""
Sep 10 15:00:54.282460 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.282340    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-b8f4d6\""
Sep 10 15:00:54.294871 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.294846    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"]
Sep 10 15:00:54.364839 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.364754    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjhmd\" (UniqueName: \"kubernetes.io/projected/84fc2303-5047-48a6-b591-673a37dd7f84-kube-api-access-qjhmd\") pod \"payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.364839 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.364809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84fc2303-5047-48a6-b591-673a37dd7f84-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.465892 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.465851    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjhmd\" (UniqueName: \"kubernetes.io/projected/84fc2303-5047-48a6-b591-673a37dd7f84-kube-api-access-qjhmd\") pod \"payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.466083 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.465909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84fc2303-5047-48a6-b591-673a37dd7f84-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.466684 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.466660    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84fc2303-5047-48a6-b591-673a37dd7f84-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.511236 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.511201    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjhmd\" (UniqueName: \"kubernetes.io/projected/84fc2303-5047-48a6-b591-673a37dd7f84-kube-api-access-qjhmd\") pod \"payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.588298 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.588255    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:54.904727 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:54.904681    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"]
Sep 10 15:00:54.906742 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:00:54.906713    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84fc2303_5047_48a6_b591_673a37dd7f84.slice/crio-5d196178ea8ace54069180260c463df5c00f47dfd31a0184fe4f45996e2049e6 WatchSource:0}: Error finding container 5d196178ea8ace54069180260c463df5c00f47dfd31a0184fe4f45996e2049e6: Status 404 returned error can't find the container with id 5d196178ea8ace54069180260c463df5c00f47dfd31a0184fe4f45996e2049e6
Sep 10 15:00:55.735945 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:55.735908    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m" event={"ID":"84fc2303-5047-48a6-b591-673a37dd7f84","Type":"ContainerStarted","Data":"0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5"}
Sep 10 15:00:55.736148 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:55.735968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m" event={"ID":"84fc2303-5047-48a6-b591-673a37dd7f84","Type":"ContainerStarted","Data":"5d196178ea8ace54069180260c463df5c00f47dfd31a0184fe4f45996e2049e6"}
Sep 10 15:00:55.736148 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:55.736095    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:00:55.781000 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:55.780940    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m" podStartSLOduration=1.78091526 podStartE2EDuration="1.78091526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:55.778055325 +0000 UTC m=+1985.133868260" watchObservedRunningTime="2026-09-10 15:00:55.78091526 +0000 UTC m=+1985.136728168"
Sep 10 15:00:57.966725 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:00:57.966686    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:01:06.743955 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:06.743922    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:01:15.664469 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:15.664428    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:01:20.825147 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:20.825104    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-bzddb_427792ba-b7e1-4f48-957c-feec063e4e34/payload-processing/0.log"
Sep 10 15:01:26.863854 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.863815    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"]
Sep 10 15:01:26.867467 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.867450    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:26.871813 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.871790    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ae0f24\"/\"openshift-service-ca.crt\""
Sep 10 15:01:26.871813 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.871812    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ae0f24\"/\"kube-root-ca.crt\""
Sep 10 15:01:26.872738 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.872710    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ae0f24\"/\"ipp-route-ae0f24-kserve-self-signed-certs\""
Sep 10 15:01:26.872858 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.872744    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ae0f24\"/\"default-dockercfg-ncw7v\""
Sep 10 15:01:26.882887 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.882861    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"]
Sep 10 15:01:26.916564 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.916532    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:01:26.965202 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.965166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-home\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:26.965407 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.965208    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-model-cache\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:26.965407 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.965230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tls-certs\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:26.965407 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.965354    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tmp-dir\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:26.965594 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.965406    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-dshm\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:26.965594 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:26.965499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrt4v\" (UniqueName: \"kubernetes.io/projected/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-kube-api-access-wrt4v\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066465 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tmp-dir\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066645 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066475    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-dshm\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066645 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrt4v\" (UniqueName: \"kubernetes.io/projected/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-kube-api-access-wrt4v\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066645 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-home\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066645 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-model-cache\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066645 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tls-certs\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066920 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066799    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tmp-dir\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.066968 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066946    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-home\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.067005 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.066986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-model-cache\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.068982 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.068958    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-dshm\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.069072 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.068987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tls-certs\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.084117 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.084082    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrt4v\" (UniqueName: \"kubernetes.io/projected/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-kube-api-access-wrt4v\") pod \"ipp-route-ae0f24-kserve-656d47fbc5-tqqzx\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") " pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.177586 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.177498    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:27.451305 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.451265    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"]
Sep 10 15:01:27.452427 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:01:27.452402    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb82af0b3_d30e_4ef7_8399_dd9e3f13b006.slice/crio-22926a615ef52f606767bd6b858573d25d080c637a2db93eafa91fb2af577e73 WatchSource:0}: Error finding container 22926a615ef52f606767bd6b858573d25d080c637a2db93eafa91fb2af577e73: Status 404 returned error can't find the container with id 22926a615ef52f606767bd6b858573d25d080c637a2db93eafa91fb2af577e73
Sep 10 15:01:27.925635 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.925544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx" event={"ID":"b82af0b3-d30e-4ef7-8399-dd9e3f13b006","Type":"ContainerStarted","Data":"5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86"}
Sep 10 15:01:27.925635 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.925600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx" event={"ID":"b82af0b3-d30e-4ef7-8399-dd9e3f13b006","Type":"ContainerStarted","Data":"22926a615ef52f606767bd6b858573d25d080c637a2db93eafa91fb2af577e73"}
Sep 10 15:01:27.942976 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:27.942930    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx" podStartSLOduration=1.942913258 podStartE2EDuration="1.942913258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:01:27.941664757 +0000 UTC m=+2017.297477687" watchObservedRunningTime="2026-09-10 15:01:27.942913258 +0000 UTC m=+2017.298726530"
Sep 10 15:01:30.788892 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:30.788841    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:01:37.178552 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:37.178515    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:37.179030 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:37.178601    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:37.191155 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:37.191133    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:37.994569 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:37.994537    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:01:38.672768 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:38.672721    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:01:38.772756 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:01:38.772723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:00.783378 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:00.783340    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:02:07.094707 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:07.094672    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2_5038e7c1-60f9-47da-8e50-aace901afcf7/payload-processing/0.log"
Sep 10 15:02:07.219115 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:07.219084    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-bzddb_427792ba-b7e1-4f48-957c-feec063e4e34/payload-processing/0.log"
Sep 10 15:02:07.625303 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:07.625254    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:07.940820 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:07.940786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:08.267452 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:08.267363    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:10.963062 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:10.963013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:11.057080 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:11.057023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:12.241568 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.241529    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"]
Sep 10 15:02:12.246158 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.246134    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.251239 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.251215    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-fd1096-serving-cert\""
Sep 10 15:02:12.259004 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.258977    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"]
Sep 10 15:02:12.279023 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.278973    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:12.288046 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.288014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4kvw\" (UniqueName: \"kubernetes.io/projected/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-kube-api-access-c4kvw\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.288165 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.288099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.288165 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.288136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.288242 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.288190    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.389567 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.389533    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.389843 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.389586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.389843 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.389644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4kvw\" (UniqueName: \"kubernetes.io/projected/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-kube-api-access-c4kvw\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.389843 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.389694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.389843 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:02:12.389787    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-serving-cert: secret "maas-api-e2e-ipp-cleanup-fd1096-serving-cert" not found
Sep 10 15:02:12.390058 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:02:12.389849    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls podName:1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4 nodeName:}" failed. No retries permitted until 2026-09-10 15:02:12.889832253 +0000 UTC m=+2062.245645139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" (UID: "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4") : secret "maas-api-e2e-ipp-cleanup-fd1096-serving-cert" not found
Sep 10 15:02:12.390447 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.390415    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.392125 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.392105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.398520 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.398499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:12.401785 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.401762    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4kvw\" (UniqueName: \"kubernetes.io/projected/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-kube-api-access-c4kvw\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.503901 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.503821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:12.786688 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.786612    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:12.879428 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.879402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:12.894326 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.894301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:12.896901 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:12.896869    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:13.160957 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:13.160865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:13.440960 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:13.440932    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"]
Sep 10 15:02:13.442472 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:02:13.442449    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d74ed4a_62bd_49e2_a98f_8b723c3b6bd4.slice/crio-1d1a7f8f8a199f688684ec294b4243d61db78d632c13aeec3f36bee7ff947dd4 WatchSource:0}: Error finding container 1d1a7f8f8a199f688684ec294b4243d61db78d632c13aeec3f36bee7ff947dd4: Status 404 returned error can't find the container with id 1d1a7f8f8a199f688684ec294b4243d61db78d632c13aeec3f36bee7ff947dd4
Sep 10 15:02:13.443933 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:13.443915    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:02:14.186572 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:14.186539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" event={"ID":"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4","Type":"ContainerStarted","Data":"2adbd86387703ef96a723ac4eaf3bc89d7cf9a237006985e722c28523bd8c183"}
Sep 10 15:02:14.186743 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:14.186591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" event={"ID":"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4","Type":"ContainerStarted","Data":"1d1a7f8f8a199f688684ec294b4243d61db78d632c13aeec3f36bee7ff947dd4"}
Sep 10 15:02:14.186743 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:14.186640    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:14.207395 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:14.207328    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" podStartSLOduration=1.7392081799999999 podStartE2EDuration="2.20727424s" totalImagesPullingTime="468.06606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:02:14.205211918 +0000 UTC m=+2063.561024845" watchObservedRunningTime="2026-09-10 15:02:14.20727424 +0000 UTC m=+2063.563087162"
Sep 10 15:02:20.197243 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:20.197214    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:20.990150 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:20.990115    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:02:35.990612 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:35.990557    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:39.791311 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:39.791255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:42.567147 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:42.567116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:43.186792 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.186762    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"]
Sep 10 15:02:43.187015 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.186974    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" podUID="1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" containerName="maas-api" containerID="cri-o://2adbd86387703ef96a723ac4eaf3bc89d7cf9a237006985e722c28523bd8c183" gracePeriod=30
Sep 10 15:02:43.230694 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.230661    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:43.329232 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.329196    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:43.352839 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.352809    2560 generic.go:356] "Generic (PLEG): container finished" podID="1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" containerID="2adbd86387703ef96a723ac4eaf3bc89d7cf9a237006985e722c28523bd8c183" exitCode=0
Sep 10 15:02:43.352999 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.352885    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" event={"ID":"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4","Type":"ContainerDied","Data":"2adbd86387703ef96a723ac4eaf3bc89d7cf9a237006985e722c28523bd8c183"}
Sep 10 15:02:43.429241 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.429200    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:43.448197 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.448133    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:43.515426 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.515384    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:43.594753 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.594714    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-openshift-service-ca\") pod \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") "
Sep 10 15:02:43.595164 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.594866    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4kvw\" (UniqueName: \"kubernetes.io/projected/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-kube-api-access-c4kvw\") pod \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") "
Sep 10 15:02:43.595164 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.594905    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls\") pod \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") "
Sep 10 15:02:43.595164 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.594927    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-metrics-certs\") pod \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\" (UID: \"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4\") "
Sep 10 15:02:43.595319 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.595152    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" (UID: "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:02:43.597120 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.597091    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" (UID: "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:02:43.597120 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.597107    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" (UID: "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:02:43.597287 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.597127    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-kube-api-access-c4kvw" (OuterVolumeSpecName: "kube-api-access-c4kvw") pod "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" (UID: "1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4"). InnerVolumeSpecName "kube-api-access-c4kvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:02:43.696609 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.696567    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4kvw\" (UniqueName: \"kubernetes.io/projected/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-kube-api-access-c4kvw\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:02:43.696609 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.696601    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:02:43.696609 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.696617    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:02:43.696849 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.696630    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:02:43.840635 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:43.840556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:44.363506 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:44.361316    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr" event={"ID":"1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4","Type":"ContainerDied","Data":"1d1a7f8f8a199f688684ec294b4243d61db78d632c13aeec3f36bee7ff947dd4"}
Sep 10 15:02:44.363506 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:44.361388    2560 scope.go:117] "RemoveContainer" containerID="2adbd86387703ef96a723ac4eaf3bc89d7cf9a237006985e722c28523bd8c183"
Sep 10 15:02:44.363506 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:44.361544    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"
Sep 10 15:02:44.384601 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:44.384575    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"]
Sep 10 15:02:44.389339 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:44.389313    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fd1096-68c5697b9b-sz6mr"]
Sep 10 15:02:45.319256 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:45.319224    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" path="/var/lib/kubelet/pods/1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4/volumes"
Sep 10 15:02:47.883058 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:47.883023    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:02:50.508520 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:50.508475    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:50.608659 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:50.608631    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:50.809322 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:50.809207    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:53.871733 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:53.871689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:02:54.661790 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:02:54.661743    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:03.060071 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:03.060035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:07.582402 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:07.582368    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:03:10.107099 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.107060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:10.137208 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.137172    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"]
Sep 10 15:03:10.137525 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.137476    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2" podUID="5038e7c1-60f9-47da-8e50-aace901afcf7" containerName="payload-processing" containerID="cri-o://b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c" gracePeriod=30
Sep 10 15:03:10.143823 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.143750    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"]
Sep 10 15:03:10.144262 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.144183    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm" podUID="7e3b266b-b308-404a-987c-ae39d3480513" containerName="payload-pre-processing" containerID="cri-o://e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75" gracePeriod=30
Sep 10 15:03:10.185426 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.185395    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:10.252679 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.252651    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:10.322330 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.321800    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:10.390663 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.390596    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:10.416067 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.416046    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:03:10.446452 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.446426    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:03:10.513703 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.513673    2560 generic.go:356] "Generic (PLEG): container finished" podID="5038e7c1-60f9-47da-8e50-aace901afcf7" containerID="b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c" exitCode=0
Sep 10 15:03:10.513870 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.513734    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"
Sep 10 15:03:10.513870 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.513769    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2" event={"ID":"5038e7c1-60f9-47da-8e50-aace901afcf7","Type":"ContainerDied","Data":"b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c"}
Sep 10 15:03:10.513870 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.513847    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2" event={"ID":"5038e7c1-60f9-47da-8e50-aace901afcf7","Type":"ContainerDied","Data":"b2129c51cfa3a13a9882bf3e49146964a70a9472409867c69e9c85eaa45bcbe2"}
Sep 10 15:03:10.513995 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.513873    2560 scope.go:117] "RemoveContainer" containerID="b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c"
Sep 10 15:03:10.515114 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.515093    2560 generic.go:356] "Generic (PLEG): container finished" podID="7e3b266b-b308-404a-987c-ae39d3480513" containerID="e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75" exitCode=0
Sep 10 15:03:10.515220 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.515161    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"
Sep 10 15:03:10.515220 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.515181    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm" event={"ID":"7e3b266b-b308-404a-987c-ae39d3480513","Type":"ContainerDied","Data":"e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75"}
Sep 10 15:03:10.515359 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.515222    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm" event={"ID":"7e3b266b-b308-404a-987c-ae39d3480513","Type":"ContainerDied","Data":"93f253eada8adcebbd1c5a0cb40dd76979ead46407fe66c3166d83517b5cc967"}
Sep 10 15:03:10.522886 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.522871    2560 scope.go:117] "RemoveContainer" containerID="b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c"
Sep 10 15:03:10.523127 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:03:10.523105    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c\": container with ID starting with b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c not found: ID does not exist" containerID="b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c"
Sep 10 15:03:10.523178 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.523135    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c"} err="failed to get container status \"b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c\": rpc error: code = NotFound desc = could not find container \"b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c\": container with ID starting with b3118369b83fcba6a89b797d950d18a605b7e173fb19a17d13245cfdc33f4b3c not found: ID does not exist"
Sep 10 15:03:10.523178 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.523163    2560 scope.go:117] "RemoveContainer" containerID="e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75"
Sep 10 15:03:10.530918 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.530903    2560 scope.go:117] "RemoveContainer" containerID="e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75"
Sep 10 15:03:10.531186 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:03:10.531165    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75\": container with ID starting with e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75 not found: ID does not exist" containerID="e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75"
Sep 10 15:03:10.531266 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.531190    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75"} err="failed to get container status \"e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75\": rpc error: code = NotFound desc = could not find container \"e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75\": container with ID starting with e3caafaaabdb4f23f32514838c16e471f46161dd47ccd46994889dc9285c4a75 not found: ID does not exist"
Sep 10 15:03:10.547602 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.547581    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcsbf\" (UniqueName: \"kubernetes.io/projected/7e3b266b-b308-404a-987c-ae39d3480513-kube-api-access-gcsbf\") pod \"7e3b266b-b308-404a-987c-ae39d3480513\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") "
Sep 10 15:03:10.547682 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.547626    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vnsc\" (UniqueName: \"kubernetes.io/projected/5038e7c1-60f9-47da-8e50-aace901afcf7-kube-api-access-4vnsc\") pod \"5038e7c1-60f9-47da-8e50-aace901afcf7\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") "
Sep 10 15:03:10.547725 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.547701    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5038e7c1-60f9-47da-8e50-aace901afcf7-plugins-config-volume\") pod \"5038e7c1-60f9-47da-8e50-aace901afcf7\" (UID: \"5038e7c1-60f9-47da-8e50-aace901afcf7\") "
Sep 10 15:03:10.547760 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.547733    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e3b266b-b308-404a-987c-ae39d3480513-plugins-config-volume\") pod \"7e3b266b-b308-404a-987c-ae39d3480513\" (UID: \"7e3b266b-b308-404a-987c-ae39d3480513\") "
Sep 10 15:03:10.548088 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.548067    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5038e7c1-60f9-47da-8e50-aace901afcf7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5038e7c1-60f9-47da-8e50-aace901afcf7" (UID: "5038e7c1-60f9-47da-8e50-aace901afcf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:03:10.548157 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.548113    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e3b266b-b308-404a-987c-ae39d3480513-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e3b266b-b308-404a-987c-ae39d3480513" (UID: "7e3b266b-b308-404a-987c-ae39d3480513"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:03:10.549609 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.549588    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e3b266b-b308-404a-987c-ae39d3480513-kube-api-access-gcsbf" (OuterVolumeSpecName: "kube-api-access-gcsbf") pod "7e3b266b-b308-404a-987c-ae39d3480513" (UID: "7e3b266b-b308-404a-987c-ae39d3480513"). InnerVolumeSpecName "kube-api-access-gcsbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:03:10.549609 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.549599    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5038e7c1-60f9-47da-8e50-aace901afcf7-kube-api-access-4vnsc" (OuterVolumeSpecName: "kube-api-access-4vnsc") pod "5038e7c1-60f9-47da-8e50-aace901afcf7" (UID: "5038e7c1-60f9-47da-8e50-aace901afcf7"). InnerVolumeSpecName "kube-api-access-4vnsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:03:10.648794 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.648720    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcsbf\" (UniqueName: \"kubernetes.io/projected/7e3b266b-b308-404a-987c-ae39d3480513-kube-api-access-gcsbf\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:10.648794 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.648749    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vnsc\" (UniqueName: \"kubernetes.io/projected/5038e7c1-60f9-47da-8e50-aace901afcf7-kube-api-access-4vnsc\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:10.648794 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.648759    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5038e7c1-60f9-47da-8e50-aace901afcf7-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:10.648794 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.648768    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e3b266b-b308-404a-987c-ae39d3480513-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:10.702384 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.702355    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:10.839275 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.839249    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"]
Sep 10 15:03:10.843718 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.843693    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ae0f24-56885f5d57-bl6c2"]
Sep 10 15:03:10.853720 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.853696    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"]
Sep 10 15:03:10.859426 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:10.859401    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ae0f24-6fcfbc9f4b-4vgtm"]
Sep 10 15:03:11.318155 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:11.318125    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5038e7c1-60f9-47da-8e50-aace901afcf7" path="/var/lib/kubelet/pods/5038e7c1-60f9-47da-8e50-aace901afcf7/volumes"
Sep 10 15:03:11.318539 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:11.318523    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e3b266b-b308-404a-987c-ae39d3480513" path="/var/lib/kubelet/pods/7e3b266b-b308-404a-987c-ae39d3480513/volumes"
Sep 10 15:03:15.680741 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:15.680701    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"]
Sep 10 15:03:15.776671 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:15.776639    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:20.901900 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:20.901862    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:03:22.574481 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:22.574444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:30.700926 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:30.700860    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx" podUID="b82af0b3-d30e-4ef7-8399-dd9e3f13b006" containerName="main" containerID="cri-o://5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86" gracePeriod=45
Sep 10 15:03:30.955571 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:30.955509    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:03:31.043560 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043527    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrt4v\" (UniqueName: \"kubernetes.io/projected/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-kube-api-access-wrt4v\") pod \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") "
Sep 10 15:03:31.043729 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043583    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tmp-dir\") pod \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") "
Sep 10 15:03:31.043729 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043615    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-home\") pod \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") "
Sep 10 15:03:31.043729 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043639    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-model-cache\") pod \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") "
Sep 10 15:03:31.043729 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043666    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-dshm\") pod \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") "
Sep 10 15:03:31.043729 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043698    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tls-certs\") pod \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\" (UID: \"b82af0b3-d30e-4ef7-8399-dd9e3f13b006\") "
Sep 10 15:03:31.044018 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043897    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-home" (OuterVolumeSpecName: "home") pod "b82af0b3-d30e-4ef7-8399-dd9e3f13b006" (UID: "b82af0b3-d30e-4ef7-8399-dd9e3f13b006"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:31.044018 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043912    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b82af0b3-d30e-4ef7-8399-dd9e3f13b006" (UID: "b82af0b3-d30e-4ef7-8399-dd9e3f13b006"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:31.044018 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.043927    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-model-cache" (OuterVolumeSpecName: "model-cache") pod "b82af0b3-d30e-4ef7-8399-dd9e3f13b006" (UID: "b82af0b3-d30e-4ef7-8399-dd9e3f13b006"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:31.044163 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.044095    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:31.044163 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.044115    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:31.044163 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.044129    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:31.045754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.045725    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b82af0b3-d30e-4ef7-8399-dd9e3f13b006" (UID: "b82af0b3-d30e-4ef7-8399-dd9e3f13b006"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:03:31.045754 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.045734    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-dshm" (OuterVolumeSpecName: "dshm") pod "b82af0b3-d30e-4ef7-8399-dd9e3f13b006" (UID: "b82af0b3-d30e-4ef7-8399-dd9e3f13b006"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:31.045905 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.045752    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-kube-api-access-wrt4v" (OuterVolumeSpecName: "kube-api-access-wrt4v") pod "b82af0b3-d30e-4ef7-8399-dd9e3f13b006" (UID: "b82af0b3-d30e-4ef7-8399-dd9e3f13b006"). InnerVolumeSpecName "kube-api-access-wrt4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:03:31.144646 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.144606    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrt4v\" (UniqueName: \"kubernetes.io/projected/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-kube-api-access-wrt4v\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:31.144646 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.144641    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:31.144846 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.144655    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b82af0b3-d30e-4ef7-8399-dd9e3f13b006-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:31.633918 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.633828    2560 generic.go:356] "Generic (PLEG): container finished" podID="b82af0b3-d30e-4ef7-8399-dd9e3f13b006" containerID="5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86" exitCode=0
Sep 10 15:03:31.633918 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.633881    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"
Sep 10 15:03:31.634152 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.633914    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx" event={"ID":"b82af0b3-d30e-4ef7-8399-dd9e3f13b006","Type":"ContainerDied","Data":"5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86"}
Sep 10 15:03:31.634152 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.633970    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx" event={"ID":"b82af0b3-d30e-4ef7-8399-dd9e3f13b006","Type":"ContainerDied","Data":"22926a615ef52f606767bd6b858573d25d080c637a2db93eafa91fb2af577e73"}
Sep 10 15:03:31.634152 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.633992    2560 scope.go:117] "RemoveContainer" containerID="5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86"
Sep 10 15:03:31.642671 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.642654    2560 scope.go:117] "RemoveContainer" containerID="5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86"
Sep 10 15:03:31.642943 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:03:31.642917    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86\": container with ID starting with 5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86 not found: ID does not exist" containerID="5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86"
Sep 10 15:03:31.643008 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.642942    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86"} err="failed to get container status \"5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86\": rpc error: code = NotFound desc = could not find container \"5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86\": container with ID starting with 5cc2b67e664026b07ec7338e0c8d22515c05c978574bab65ef87038e38432f86 not found: ID does not exist"
Sep 10 15:03:31.653266 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.653243    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"]
Sep 10 15:03:31.658666 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:31.658646    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-ae0f24/ipp-route-ae0f24-kserve-656d47fbc5-tqqzx"]
Sep 10 15:03:33.319020 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:33.318985    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82af0b3-d30e-4ef7-8399-dd9e3f13b006" path="/var/lib/kubelet/pods/b82af0b3-d30e-4ef7-8399-dd9e3f13b006/volumes"
Sep 10 15:03:53.666902 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:53.666869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:03:55.440081 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:03:55.440048    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:01.517457 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:01.517428    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:01.614435 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:01.614400    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:01.848349 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:01.848209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:02.375496 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:02.375463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.040840 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.040801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.105114 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.105077    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"]
Sep 10 15:04:13.105371 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.105344    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m" podUID="84fc2303-5047-48a6-b591-673a37dd7f84" containerName="payload-pre-processing" containerID="cri-o://0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5" gracePeriod=30
Sep 10 15:04:13.190181 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.190137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.306835 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.306764    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.367941 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.367918    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:04:13.417108 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.417076    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.425177 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.425155    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjhmd\" (UniqueName: \"kubernetes.io/projected/84fc2303-5047-48a6-b591-673a37dd7f84-kube-api-access-qjhmd\") pod \"84fc2303-5047-48a6-b591-673a37dd7f84\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") "
Sep 10 15:04:13.425311 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.425183    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84fc2303-5047-48a6-b591-673a37dd7f84-plugins-config-volume\") pod \"84fc2303-5047-48a6-b591-673a37dd7f84\" (UID: \"84fc2303-5047-48a6-b591-673a37dd7f84\") "
Sep 10 15:04:13.425712 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.425684    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84fc2303-5047-48a6-b591-673a37dd7f84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84fc2303-5047-48a6-b591-673a37dd7f84" (UID: "84fc2303-5047-48a6-b591-673a37dd7f84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:04:13.427378 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.427359    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84fc2303-5047-48a6-b591-673a37dd7f84-kube-api-access-qjhmd" (OuterVolumeSpecName: "kube-api-access-qjhmd") pod "84fc2303-5047-48a6-b591-673a37dd7f84" (UID: "84fc2303-5047-48a6-b591-673a37dd7f84"). InnerVolumeSpecName "kube-api-access-qjhmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:04:13.526624 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.526591    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjhmd\" (UniqueName: \"kubernetes.io/projected/84fc2303-5047-48a6-b591-673a37dd7f84-kube-api-access-qjhmd\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:13.526624 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.526620    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84fc2303-5047-48a6-b591-673a37dd7f84-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:13.567790 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.567715    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.816817 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.816785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:13.879086 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.879004    2560 generic.go:356] "Generic (PLEG): container finished" podID="84fc2303-5047-48a6-b591-673a37dd7f84" containerID="0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5" exitCode=0
Sep 10 15:04:13.879086 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.879052    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"
Sep 10 15:04:13.879263 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.879095    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m" event={"ID":"84fc2303-5047-48a6-b591-673a37dd7f84","Type":"ContainerDied","Data":"0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5"}
Sep 10 15:04:13.879263 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.879159    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m" event={"ID":"84fc2303-5047-48a6-b591-673a37dd7f84","Type":"ContainerDied","Data":"5d196178ea8ace54069180260c463df5c00f47dfd31a0184fe4f45996e2049e6"}
Sep 10 15:04:13.879263 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.879189    2560 scope.go:117] "RemoveContainer" containerID="0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5"
Sep 10 15:04:13.888364 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.888345    2560 scope.go:117] "RemoveContainer" containerID="0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5"
Sep 10 15:04:13.888631 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:04:13.888608    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5\": container with ID starting with 0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5 not found: ID does not exist" containerID="0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5"
Sep 10 15:04:13.888679 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.888638    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5"} err="failed to get container status \"0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5\": rpc error: code = NotFound desc = could not find container \"0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5\": container with ID starting with 0855025de989dfc58805d04623d360025df7a103b760d19384e3dc53124b64d5 not found: ID does not exist"
Sep 10 15:04:13.904354 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.904327    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"]
Sep 10 15:04:13.910798 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:13.910779    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b8f4d6-5485cc5566-shl2m"]
Sep 10 15:04:15.319029 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:15.318993    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84fc2303-5047-48a6-b591-673a37dd7f84" path="/var/lib/kubelet/pods/84fc2303-5047-48a6-b591-673a37dd7f84/volumes"
Sep 10 15:04:16.343116 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:16.343082    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:04:21.246098 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:21.246061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:21.345942 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:21.345911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:21.711320 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:21.711267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:22.312274 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:22.312241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:32.835084 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.835049    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"]
Sep 10 15:04:32.835523 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.835271    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" podUID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" containerName="maas-api" containerID="cri-o://391ec1932ac93b6cfbe123d574454831f329ce570aa6b50836e841e07d711c81" gracePeriod=30
Sep 10 15:04:32.853064 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.853025    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" podUID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.77:8443/health\": dial tcp 10.132.0.77:8443: connect: connection refused"
Sep 10 15:04:32.978726 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.978674    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:32.985100 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.985045    2560 generic.go:356] "Generic (PLEG): container finished" podID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" containerID="391ec1932ac93b6cfbe123d574454831f329ce570aa6b50836e841e07d711c81" exitCode=0
Sep 10 15:04:32.985300 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.985239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" event={"ID":"c4bd9805-96e2-4700-a5f2-f6ced2d20048","Type":"ContainerDied","Data":"391ec1932ac93b6cfbe123d574454831f329ce570aa6b50836e841e07d711c81"}
Sep 10 15:04:32.995442 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.995412    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"]
Sep 10 15:04:32.995614 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:32.995593    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4" podUID="5334ba06-f4c5-4c5c-8e07-ecfedaa33039" containerName="payload-processing" containerID="cri-o://c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f" gracePeriod=30
Sep 10 15:04:33.120583 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.120562    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 15:04:33.216480 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.216437    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bd9805-96e2-4700-a5f2-f6ced2d20048-openshift-service-ca\") pod \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") "
Sep 10 15:04:33.216633 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.216528    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-metrics-certs\") pod \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") "
Sep 10 15:04:33.216689 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.216648    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls\") pod \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") "
Sep 10 15:04:33.216813 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.216795    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lcnj\" (UniqueName: \"kubernetes.io/projected/c4bd9805-96e2-4700-a5f2-f6ced2d20048-kube-api-access-5lcnj\") pod \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\" (UID: \"c4bd9805-96e2-4700-a5f2-f6ced2d20048\") "
Sep 10 15:04:33.217166 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.216993    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bd9805-96e2-4700-a5f2-f6ced2d20048-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4bd9805-96e2-4700-a5f2-f6ced2d20048" (UID: "c4bd9805-96e2-4700-a5f2-f6ced2d20048"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:04:33.217166 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.217147    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bd9805-96e2-4700-a5f2-f6ced2d20048-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:33.219519 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.219484    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4bd9805-96e2-4700-a5f2-f6ced2d20048" (UID: "c4bd9805-96e2-4700-a5f2-f6ced2d20048"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:04:33.219627 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.219581    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bd9805-96e2-4700-a5f2-f6ced2d20048-kube-api-access-5lcnj" (OuterVolumeSpecName: "kube-api-access-5lcnj") pod "c4bd9805-96e2-4700-a5f2-f6ced2d20048" (UID: "c4bd9805-96e2-4700-a5f2-f6ced2d20048"). InnerVolumeSpecName "kube-api-access-5lcnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:04:33.219627 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.219588    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c4bd9805-96e2-4700-a5f2-f6ced2d20048" (UID: "c4bd9805-96e2-4700-a5f2-f6ced2d20048"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:04:33.222556 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.222532    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:33.245724 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.245705    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 15:04:33.317961 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.317920    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-plugins-config-volume\") pod \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") "
Sep 10 15:04:33.318137 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.317976    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svm4j\" (UniqueName: \"kubernetes.io/projected/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-kube-api-access-svm4j\") pod \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\" (UID: \"5334ba06-f4c5-4c5c-8e07-ecfedaa33039\") "
Sep 10 15:04:33.318303 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.318257    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lcnj\" (UniqueName: \"kubernetes.io/projected/c4bd9805-96e2-4700-a5f2-f6ced2d20048-kube-api-access-5lcnj\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:33.318399 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.318298    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:33.318399 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.318323    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4bd9805-96e2-4700-a5f2-f6ced2d20048-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:33.318485 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.318389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5334ba06-f4c5-4c5c-8e07-ecfedaa33039" (UID: "5334ba06-f4c5-4c5c-8e07-ecfedaa33039"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:04:33.320559 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.320536    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-kube-api-access-svm4j" (OuterVolumeSpecName: "kube-api-access-svm4j") pod "5334ba06-f4c5-4c5c-8e07-ecfedaa33039" (UID: "5334ba06-f4c5-4c5c-8e07-ecfedaa33039"). InnerVolumeSpecName "kube-api-access-svm4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:04:33.373552 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.373492    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:33.419710 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.419678    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-plugins-config-volume\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:33.419710 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.419705    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svm4j\" (UniqueName: \"kubernetes.io/projected/5334ba06-f4c5-4c5c-8e07-ecfedaa33039-kube-api-access-svm4j\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:33.534132 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.534099    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:33.859788 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.859758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:33.991455 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.991424    2560 generic.go:356] "Generic (PLEG): container finished" podID="5334ba06-f4c5-4c5c-8e07-ecfedaa33039" containerID="c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f" exitCode=0
Sep 10 15:04:33.991633 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.991471    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"
Sep 10 15:04:33.991633 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.991517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4" event={"ID":"5334ba06-f4c5-4c5c-8e07-ecfedaa33039","Type":"ContainerDied","Data":"c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f"}
Sep 10 15:04:33.991633 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.991571    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4" event={"ID":"5334ba06-f4c5-4c5c-8e07-ecfedaa33039","Type":"ContainerDied","Data":"a96d5bcf1153abcad75284545df85c7e850be6e19a365f1c8c9d038ec2819e44"}
Sep 10 15:04:33.991633 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.991596    2560 scope.go:117] "RemoveContainer" containerID="c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f"
Sep 10 15:04:33.993199 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.993182    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"
Sep 10 15:04:33.993293 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:33.993197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg" event={"ID":"c4bd9805-96e2-4700-a5f2-f6ced2d20048","Type":"ContainerDied","Data":"7ac9df9b96321a3f77fd50aca15b46203d35c35106aad2b9238d166dbdb8f7b5"}
Sep 10 15:04:34.000905 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.000889    2560 scope.go:117] "RemoveContainer" containerID="c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f"
Sep 10 15:04:34.001169 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:04:34.001150    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f\": container with ID starting with c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f not found: ID does not exist" containerID="c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f"
Sep 10 15:04:34.001223 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.001178    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f"} err="failed to get container status \"c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f\": rpc error: code = NotFound desc = could not find container \"c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f\": container with ID starting with c1f97371fce68c12e91d2cdac14bac4af7ddb7d9de9765365f2298dc59ffca2f not found: ID does not exist"
Sep 10 15:04:34.001223 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.001195    2560 scope.go:117] "RemoveContainer" containerID="391ec1932ac93b6cfbe123d574454831f329ce570aa6b50836e841e07d711c81"
Sep 10 15:04:34.018331 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.018299    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"]
Sep 10 15:04:34.021123 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.021101    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-12b6f6-64c7ff75d7-ftvmg"]
Sep 10 15:04:34.033668 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.033647    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"]
Sep 10 15:04:34.042845 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:34.042822    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-12b6f6-78768f788f-bjnd4"]
Sep 10 15:04:35.319228 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:35.319192    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5334ba06-f4c5-4c5c-8e07-ecfedaa33039" path="/var/lib/kubelet/pods/5334ba06-f4c5-4c5c-8e07-ecfedaa33039/volumes"
Sep 10 15:04:35.319670 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:35.319652    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" path="/var/lib/kubelet/pods/c4bd9805-96e2-4700-a5f2-f6ced2d20048/volumes"
Sep 10 15:04:35.992360 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:35.992318    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:04:40.281953 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:40.281922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:40.402062 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:40.402029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:40.673102 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:40.673022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:41.235520 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:41.235482    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:52.189472 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:52.189438    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:52.431590 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:52.431538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:52.860573 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:52.860545    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:53.196163 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:53.196130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:53.541619 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:53.541539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:04:55.167098 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:04:55.167067    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:05:01.342925 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:01.342884    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:05:01.518054 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:01.518014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:05:01.807866 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:01.807836    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:05:04.899563 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:04.899525    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-srtf8"]
Sep 10 15:05:04.900033 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:04.899756    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b457c4dbd-srtf8" podUID="6b5488a8-6dc5-436e-a86f-72c3ee732e06" containerName="manager" containerID="cri-o://53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14" gracePeriod=30
Sep 10 15:05:05.151660 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.151595    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 15:05:05.173340 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.173310    2560 generic.go:356] "Generic (PLEG): container finished" podID="6b5488a8-6dc5-436e-a86f-72c3ee732e06" containerID="53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14" exitCode=0
Sep 10 15:05:05.173504 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.173353    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-srtf8"
Sep 10 15:05:05.173504 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.173371    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-srtf8" event={"ID":"6b5488a8-6dc5-436e-a86f-72c3ee732e06","Type":"ContainerDied","Data":"53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14"}
Sep 10 15:05:05.173504 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.173413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-srtf8" event={"ID":"6b5488a8-6dc5-436e-a86f-72c3ee732e06","Type":"ContainerDied","Data":"001ebf17fb99bf3671d49346144e3214b24e1cebb731720d7aa223b9ca0a9b4f"}
Sep 10 15:05:05.173504 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.173435    2560 scope.go:117] "RemoveContainer" containerID="53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14"
Sep 10 15:05:05.182699 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.182678    2560 scope.go:117] "RemoveContainer" containerID="53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14"
Sep 10 15:05:05.182976 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:05:05.182954    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14\": container with ID starting with 53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14 not found: ID does not exist" containerID="53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14"
Sep 10 15:05:05.183059 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.182988    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14"} err="failed to get container status \"53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14\": rpc error: code = NotFound desc = could not find container \"53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14\": container with ID starting with 53d10f5245c33e821a10819ad9a692b9f05ff2d9f06c12c71a474b04e260ee14 not found: ID does not exist"
Sep 10 15:05:05.194435 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.194407    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-webhook-cert\") pod \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") "
Sep 10 15:05:05.194580 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.194455    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm24r\" (UniqueName: \"kubernetes.io/projected/6b5488a8-6dc5-436e-a86f-72c3ee732e06-kube-api-access-sm24r\") pod \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") "
Sep 10 15:05:05.194580 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.194525    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-metrics-certs\") pod \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\" (UID: \"6b5488a8-6dc5-436e-a86f-72c3ee732e06\") "
Sep 10 15:05:05.196641 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.196606    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b5488a8-6dc5-436e-a86f-72c3ee732e06" (UID: "6b5488a8-6dc5-436e-a86f-72c3ee732e06"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:05:05.196641 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.196623    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6b5488a8-6dc5-436e-a86f-72c3ee732e06" (UID: "6b5488a8-6dc5-436e-a86f-72c3ee732e06"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:05:05.196803 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.196719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5488a8-6dc5-436e-a86f-72c3ee732e06-kube-api-access-sm24r" (OuterVolumeSpecName: "kube-api-access-sm24r") pod "6b5488a8-6dc5-436e-a86f-72c3ee732e06" (UID: "6b5488a8-6dc5-436e-a86f-72c3ee732e06"). InnerVolumeSpecName "kube-api-access-sm24r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:05.295900 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.295845    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm24r\" (UniqueName: \"kubernetes.io/projected/6b5488a8-6dc5-436e-a86f-72c3ee732e06-kube-api-access-sm24r\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:05.295900 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.295897    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:05.295900 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.295907    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b5488a8-6dc5-436e-a86f-72c3ee732e06-webhook-cert\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:05.497514 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.497481    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-srtf8"]
Sep 10 15:05:05.499574 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:05.499551    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-srtf8"]
Sep 10 15:05:06.147459 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:06.147422    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:05:07.318290 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:07.318252    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b5488a8-6dc5-436e-a86f-72c3ee732e06" path="/var/lib/kubelet/pods/6b5488a8-6dc5-436e-a86f-72c3ee732e06/volumes"
Sep 10 15:05:31.128815 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:31.128781    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:05:31.437848 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:05:31.437810    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:06:04.355417 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:04.355383    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:06:06.375960 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.375922    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-z862t"]
Sep 10 15:06:06.376461 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376442    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84fc2303-5047-48a6-b591-673a37dd7f84" containerName="payload-pre-processing"
Sep 10 15:06:06.376461 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376462    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="84fc2303-5047-48a6-b591-673a37dd7f84" containerName="payload-pre-processing"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376477    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" containerName="maas-api"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376484    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" containerName="maas-api"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376493    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e3b266b-b308-404a-987c-ae39d3480513" containerName="payload-pre-processing"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376498    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e3b266b-b308-404a-987c-ae39d3480513" containerName="payload-pre-processing"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376504    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b82af0b3-d30e-4ef7-8399-dd9e3f13b006" containerName="main"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376510    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b82af0b3-d30e-4ef7-8399-dd9e3f13b006" containerName="main"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376518    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5038e7c1-60f9-47da-8e50-aace901afcf7" containerName="payload-processing"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376524    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5038e7c1-60f9-47da-8e50-aace901afcf7" containerName="payload-processing"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376533    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" containerName="maas-api"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376538    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" containerName="maas-api"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376546    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5334ba06-f4c5-4c5c-8e07-ecfedaa33039" containerName="payload-processing"
Sep 10 15:06:06.376549 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376552    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5334ba06-f4c5-4c5c-8e07-ecfedaa33039" containerName="payload-processing"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376562    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5488a8-6dc5-436e-a86f-72c3ee732e06" containerName="manager"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376570    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5488a8-6dc5-436e-a86f-72c3ee732e06" containerName="manager"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376648    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b82af0b3-d30e-4ef7-8399-dd9e3f13b006" containerName="main"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376660    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4bd9805-96e2-4700-a5f2-f6ced2d20048" containerName="maas-api"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376669    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d74ed4a-62bd-49e2-a98f-8b723c3b6bd4" containerName="maas-api"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376679    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="84fc2303-5047-48a6-b591-673a37dd7f84" containerName="payload-pre-processing"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376691    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5038e7c1-60f9-47da-8e50-aace901afcf7" containerName="payload-processing"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376701    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e3b266b-b308-404a-987c-ae39d3480513" containerName="payload-pre-processing"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376710    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b5488a8-6dc5-436e-a86f-72c3ee732e06" containerName="manager"
Sep 10 15:06:06.376932 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.376718    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5334ba06-f4c5-4c5c-8e07-ecfedaa33039" containerName="payload-processing"
Sep 10 15:06:06.379994 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.379976    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.397049 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.397014    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:06:06.397172 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.397077    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:06:06.397172 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.397096    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6hmtm\""
Sep 10 15:06:06.397172 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.397096    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:06:06.408777 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.408751    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-z862t"]
Sep 10 15:06:06.441625 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.441592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-metrics-certs\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.441775 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.441641    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-webhook-cert\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.441775 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.441673    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dvmn\" (UniqueName: \"kubernetes.io/projected/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-kube-api-access-6dvmn\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.542330 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.542269    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-metrics-certs\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.542330 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.542341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-webhook-cert\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.542566 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.542362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dvmn\" (UniqueName: \"kubernetes.io/projected/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-kube-api-access-6dvmn\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.544788 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.544755    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-webhook-cert\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.544788 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.544784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-metrics-certs\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.553222 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.553200    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dvmn\" (UniqueName: \"kubernetes.io/projected/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-kube-api-access-6dvmn\") pod \"maas-controller-7b457c4dbd-z862t\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") " pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.692063 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.692033    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:06.977631 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:06.977603    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-z862t"]
Sep 10 15:06:06.979260 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:06:06.979231    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc800737b_7bc9_43eb_8e93_54edfa9b1ba8.slice/crio-8a501c4bc362e139ac075198949cbc5c31beb1f2e8bd830595a3f6a8232ebfe9 WatchSource:0}: Error finding container 8a501c4bc362e139ac075198949cbc5c31beb1f2e8bd830595a3f6a8232ebfe9: Status 404 returned error can't find the container with id 8a501c4bc362e139ac075198949cbc5c31beb1f2e8bd830595a3f6a8232ebfe9
Sep 10 15:06:07.510759 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:07.510723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-z862t" event={"ID":"c800737b-7bc9-43eb-8e93-54edfa9b1ba8","Type":"ContainerStarted","Data":"8a501c4bc362e139ac075198949cbc5c31beb1f2e8bd830595a3f6a8232ebfe9"}
Sep 10 15:06:08.518579 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:08.518544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-z862t" event={"ID":"c800737b-7bc9-43eb-8e93-54edfa9b1ba8","Type":"ContainerStarted","Data":"1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb"}
Sep 10 15:06:08.518953 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:08.518688    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:08.537354 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:08.537307    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b457c4dbd-z862t" podStartSLOduration=1.531216563 podStartE2EDuration="2.537273229s" totalImagesPullingTime="1.006056666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:06:08.534982427 +0000 UTC m=+2297.890795333" watchObservedRunningTime="2026-09-10 15:06:08.537273229 +0000 UTC m=+2297.893086167"
Sep 10 15:06:19.526337 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:19.526297    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:26.166551 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:26.166506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:06:26.795703 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:26.795667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:06:26.863200 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:26.863130    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-z862t"]
Sep 10 15:06:26.863570 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:26.863530    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b457c4dbd-z862t" podUID="c800737b-7bc9-43eb-8e93-54edfa9b1ba8" containerName="manager" containerID="cri-o://1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb" gracePeriod=30
Sep 10 15:06:27.100112 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.100086    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:06:27.114486 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.114462    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:27.233176 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.233144    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-webhook-cert\") pod \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") "
Sep 10 15:06:27.233652 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.233241    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-metrics-certs\") pod \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") "
Sep 10 15:06:27.233652 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.233329    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dvmn\" (UniqueName: \"kubernetes.io/projected/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-kube-api-access-6dvmn\") pod \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\" (UID: \"c800737b-7bc9-43eb-8e93-54edfa9b1ba8\") "
Sep 10 15:06:27.235462 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.235425    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-kube-api-access-6dvmn" (OuterVolumeSpecName: "kube-api-access-6dvmn") pod "c800737b-7bc9-43eb-8e93-54edfa9b1ba8" (UID: "c800737b-7bc9-43eb-8e93-54edfa9b1ba8"). InnerVolumeSpecName "kube-api-access-6dvmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:06:27.235462 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.235442    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c800737b-7bc9-43eb-8e93-54edfa9b1ba8" (UID: "c800737b-7bc9-43eb-8e93-54edfa9b1ba8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:06:27.235629 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.235510    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c800737b-7bc9-43eb-8e93-54edfa9b1ba8" (UID: "c800737b-7bc9-43eb-8e93-54edfa9b1ba8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:06:27.334462 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.334437    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:27.334462 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.334463    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dvmn\" (UniqueName: \"kubernetes.io/projected/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-kube-api-access-6dvmn\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:27.334601 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.334473    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c800737b-7bc9-43eb-8e93-54edfa9b1ba8-webhook-cert\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:27.627580 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.627547    2560 generic.go:356] "Generic (PLEG): container finished" podID="c800737b-7bc9-43eb-8e93-54edfa9b1ba8" containerID="1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb" exitCode=0
Sep 10 15:06:27.627783 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.627610    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-z862t"
Sep 10 15:06:27.627783 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.627638    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-z862t" event={"ID":"c800737b-7bc9-43eb-8e93-54edfa9b1ba8","Type":"ContainerDied","Data":"1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb"}
Sep 10 15:06:27.627783 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.627703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-z862t" event={"ID":"c800737b-7bc9-43eb-8e93-54edfa9b1ba8","Type":"ContainerDied","Data":"8a501c4bc362e139ac075198949cbc5c31beb1f2e8bd830595a3f6a8232ebfe9"}
Sep 10 15:06:27.627783 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.627733    2560 scope.go:117] "RemoveContainer" containerID="1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb"
Sep 10 15:06:27.636160 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.636141    2560 scope.go:117] "RemoveContainer" containerID="1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb"
Sep 10 15:06:27.636459 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:06:27.636436    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb\": container with ID starting with 1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb not found: ID does not exist" containerID="1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb"
Sep 10 15:06:27.636555 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.636464    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb"} err="failed to get container status \"1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb\": rpc error: code = NotFound desc = could not find container \"1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb\": container with ID starting with 1e05690d8571768333009548491ffd2be87f3f0d1297b57a46d1ab7460addacb not found: ID does not exist"
Sep 10 15:06:27.652122 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.652096    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-z862t"]
Sep 10 15:06:27.656119 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:27.656099    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-z862t"]
Sep 10 15:06:28.446559 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.446524    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:06:28.566144 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.566106    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-45ppx"]
Sep 10 15:06:28.566626 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.566604    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c800737b-7bc9-43eb-8e93-54edfa9b1ba8" containerName="manager"
Sep 10 15:06:28.566626 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.566627    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c800737b-7bc9-43eb-8e93-54edfa9b1ba8" containerName="manager"
Sep 10 15:06:28.566796 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.566732    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c800737b-7bc9-43eb-8e93-54edfa9b1ba8" containerName="manager"
Sep 10 15:06:28.571183 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.571160    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.574428 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.574405    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:06:28.574561 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.574463    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:06:28.574561 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.574524    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:06:28.574561 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.574466    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6hmtm\""
Sep 10 15:06:28.581819 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.581798    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-45ppx"]
Sep 10 15:06:28.644737 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.644704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2hxd\" (UniqueName: \"kubernetes.io/projected/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-kube-api-access-r2hxd\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.644914 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.644746    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-webhook-cert\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.644914 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.644770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-metrics-certs\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.746050 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.746017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2hxd\" (UniqueName: \"kubernetes.io/projected/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-kube-api-access-r2hxd\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.746199 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.746073    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-webhook-cert\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.746199 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.746116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-metrics-certs\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.748614 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.748589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-metrics-certs\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.748731 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.748622    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-webhook-cert\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.757647 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.757626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2hxd\" (UniqueName: \"kubernetes.io/projected/c4578920-1c2c-4bcd-b188-c55e4a89cbc3-kube-api-access-r2hxd\") pod \"maas-controller-7b457c4dbd-45ppx\" (UID: \"c4578920-1c2c-4bcd-b188-c55e4a89cbc3\") " pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:28.882450 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:28.882412    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:29.173575 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:29.173543    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b457c4dbd-45ppx"]
Sep 10 15:06:29.175209 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:06:29.175184    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4578920_1c2c_4bcd_b188_c55e4a89cbc3.slice/crio-f33cda93da183e3d4914ab440f668d633e858cbff4fec9b8fa9e0fead415644d WatchSource:0}: Error finding container f33cda93da183e3d4914ab440f668d633e858cbff4fec9b8fa9e0fead415644d: Status 404 returned error can't find the container with id f33cda93da183e3d4914ab440f668d633e858cbff4fec9b8fa9e0fead415644d
Sep 10 15:06:29.318766 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:29.318735    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c800737b-7bc9-43eb-8e93-54edfa9b1ba8" path="/var/lib/kubelet/pods/c800737b-7bc9-43eb-8e93-54edfa9b1ba8/volumes"
Sep 10 15:06:29.640971 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:29.640940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-45ppx" event={"ID":"c4578920-1c2c-4bcd-b188-c55e4a89cbc3","Type":"ContainerStarted","Data":"f33cda93da183e3d4914ab440f668d633e858cbff4fec9b8fa9e0fead415644d"}
Sep 10 15:06:30.647879 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:30.647841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b457c4dbd-45ppx" event={"ID":"c4578920-1c2c-4bcd-b188-c55e4a89cbc3","Type":"ContainerStarted","Data":"f63dd748fe946d3d18bf6ef086465c4319f2e287dbe5b8beb09da372f30bc22b"}
Sep 10 15:06:30.648264 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:30.647904    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:30.673190 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:30.673135    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b457c4dbd-45ppx" podStartSLOduration=2.145959191 podStartE2EDuration="2.673118839s" totalImagesPullingTime="527.159648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:06:30.672044881 +0000 UTC m=+2320.027857788" watchObservedRunningTime="2026-09-10 15:06:30.673118839 +0000 UTC m=+2320.028931746"
Sep 10 15:06:41.655977 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:41.655939    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b457c4dbd-45ppx"
Sep 10 15:06:54.318086 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:06:54.318048    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:07:02.446105 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:02.446065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:07:06.345216 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:06.345172    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:07:08.995930 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:08.995889    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:07:15.320072 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:15.320032    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:07:29.795114 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:29.795075    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:07:37.599333 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:37.599270    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:07:38.515919 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:38.515887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:07:39.725544 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:07:39.725506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:08:18.096590 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:08:18.096507    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:08:18.195434 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:08:18.195400    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:09:05.192248 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:05.192211    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:09:11.999171 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:11.999139    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:09:21.694064 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:21.694025    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:09:28.932320 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:28.932273    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"]
Sep 10 15:09:28.936007 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:28.935983    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:28.938569 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:28.938540    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ac593d41-kserve-self-signed-certs\""
Sep 10 15:09:28.948001 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:28.947976    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"]
Sep 10 15:09:28.972815 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:28.972737    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:09:29.025253 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.025221    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-home\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.025433 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.025261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbknv\" (UniqueName: \"kubernetes.io/projected/6f690402-36c6-40f0-b180-0c88a488b1e3-kube-api-access-qbknv\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.025433 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.025313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-dshm\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.025433 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.025368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-tmp-dir\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.025433 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.025383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f690402-36c6-40f0-b180-0c88a488b1e3-tls-certs\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.025605 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.025472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-model-cache\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.126869 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.126830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-home\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.126869 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.126882    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbknv\" (UniqueName: \"kubernetes.io/projected/6f690402-36c6-40f0-b180-0c88a488b1e3-kube-api-access-qbknv\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127102 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.126936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-dshm\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127102 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.126988    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-tmp-dir\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127102 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.127004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f690402-36c6-40f0-b180-0c88a488b1e3-tls-certs\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127102 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.127047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-model-cache\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127491 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.127470    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-tmp-dir\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127571 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.127508    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-model-cache\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.127571 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.127548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-home\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.129380 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.129358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-dshm\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.129737 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.129717    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f690402-36c6-40f0-b180-0c88a488b1e3-tls-certs\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.139204 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.139179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbknv\" (UniqueName: \"kubernetes.io/projected/6f690402-36c6-40f0-b180-0c88a488b1e3-kube-api-access-qbknv\") pod \"e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") " pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.247480 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.247389    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:29.524464 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.524431    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"]
Sep 10 15:09:29.525708 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:09:29.525679    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f690402_36c6_40f0_b180_0c88a488b1e3.slice/crio-e715fe9442e83447dd786cb58eef3d062e3f73298c85a0700eec1043489368ca WatchSource:0}: Error finding container e715fe9442e83447dd786cb58eef3d062e3f73298c85a0700eec1043489368ca: Status 404 returned error can't find the container with id e715fe9442e83447dd786cb58eef3d062e3f73298c85a0700eec1043489368ca
Sep 10 15:09:29.633364 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.633323    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx" event={"ID":"6f690402-36c6-40f0-b180-0c88a488b1e3","Type":"ContainerStarted","Data":"af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0"}
Sep 10 15:09:29.633511 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.633379    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx" event={"ID":"6f690402-36c6-40f0-b180-0c88a488b1e3","Type":"ContainerStarted","Data":"e715fe9442e83447dd786cb58eef3d062e3f73298c85a0700eec1043489368ca"}
Sep 10 15:09:29.652617 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:29.652560    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx" podStartSLOduration=1.6525431369999999 podStartE2EDuration="1.652543137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:09:29.649907001 +0000 UTC m=+2499.005719926" watchObservedRunningTime="2026-09-10 15:09:29.652543137 +0000 UTC m=+2499.008356043"
Sep 10 15:09:31.579643 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:31.579606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:09:39.248540 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:39.248504    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:39.249022 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:39.248564    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:39.261407 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:39.261384    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:39.698891 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:39.698861    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:09:43.868567 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:43.868494    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:09:47.280792 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:09:47.280749    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:15.344099 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:15.344062    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"]
Sep 10 15:10:15.365983 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:15.365951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:10:17.270463 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:17.270431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:10:17.400165 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:17.400133    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:26.584033 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:26.583998    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:30.364488 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.364444    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx" podUID="6f690402-36c6-40f0-b180-0c88a488b1e3" containerName="main" containerID="cri-o://af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0" gracePeriod=45
Sep 10 15:10:30.614939 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.614882    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:10:30.687295 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687245    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f690402-36c6-40f0-b180-0c88a488b1e3-tls-certs\") pod \"6f690402-36c6-40f0-b180-0c88a488b1e3\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") "
Sep 10 15:10:30.687295 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687298    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-home\") pod \"6f690402-36c6-40f0-b180-0c88a488b1e3\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") "
Sep 10 15:10:30.687539 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687341    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-dshm\") pod \"6f690402-36c6-40f0-b180-0c88a488b1e3\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") "
Sep 10 15:10:30.687539 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687426    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbknv\" (UniqueName: \"kubernetes.io/projected/6f690402-36c6-40f0-b180-0c88a488b1e3-kube-api-access-qbknv\") pod \"6f690402-36c6-40f0-b180-0c88a488b1e3\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") "
Sep 10 15:10:30.687539 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687482    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-model-cache\") pod \"6f690402-36c6-40f0-b180-0c88a488b1e3\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") "
Sep 10 15:10:30.687539 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687515    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-tmp-dir\") pod \"6f690402-36c6-40f0-b180-0c88a488b1e3\" (UID: \"6f690402-36c6-40f0-b180-0c88a488b1e3\") "
Sep 10 15:10:30.687764 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687615    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-home" (OuterVolumeSpecName: "home") pod "6f690402-36c6-40f0-b180-0c88a488b1e3" (UID: "6f690402-36c6-40f0-b180-0c88a488b1e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:30.687825 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687795    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f690402-36c6-40f0-b180-0c88a488b1e3" (UID: "6f690402-36c6-40f0-b180-0c88a488b1e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:30.687825 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687815    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-home\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:30.687927 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.687826    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f690402-36c6-40f0-b180-0c88a488b1e3" (UID: "6f690402-36c6-40f0-b180-0c88a488b1e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:30.689694 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.689662    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f690402-36c6-40f0-b180-0c88a488b1e3-kube-api-access-qbknv" (OuterVolumeSpecName: "kube-api-access-qbknv") pod "6f690402-36c6-40f0-b180-0c88a488b1e3" (UID: "6f690402-36c6-40f0-b180-0c88a488b1e3"). InnerVolumeSpecName "kube-api-access-qbknv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:10:30.689694 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.689684    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-dshm" (OuterVolumeSpecName: "dshm") pod "6f690402-36c6-40f0-b180-0c88a488b1e3" (UID: "6f690402-36c6-40f0-b180-0c88a488b1e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:30.689843 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.689704    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f690402-36c6-40f0-b180-0c88a488b1e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f690402-36c6-40f0-b180-0c88a488b1e3" (UID: "6f690402-36c6-40f0-b180-0c88a488b1e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:10:30.788618 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.788562    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbknv\" (UniqueName: \"kubernetes.io/projected/6f690402-36c6-40f0-b180-0c88a488b1e3-kube-api-access-qbknv\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:30.788618 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.788612    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-model-cache\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:30.788618 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.788625    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-tmp-dir\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:30.788867 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.788638    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f690402-36c6-40f0-b180-0c88a488b1e3-tls-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:30.788867 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.788649    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f690402-36c6-40f0-b180-0c88a488b1e3-dshm\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:30.969825 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.969789    2560 generic.go:356] "Generic (PLEG): container finished" podID="6f690402-36c6-40f0-b180-0c88a488b1e3" containerID="af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0" exitCode=0
Sep 10 15:10:30.970001 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.969842    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"
Sep 10 15:10:30.970001 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.969879    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx" event={"ID":"6f690402-36c6-40f0-b180-0c88a488b1e3","Type":"ContainerDied","Data":"af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0"}
Sep 10 15:10:30.970001 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.969930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx" event={"ID":"6f690402-36c6-40f0-b180-0c88a488b1e3","Type":"ContainerDied","Data":"e715fe9442e83447dd786cb58eef3d062e3f73298c85a0700eec1043489368ca"}
Sep 10 15:10:30.970001 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.969950    2560 scope.go:117] "RemoveContainer" containerID="af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0"
Sep 10 15:10:30.981245 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.981228    2560 scope.go:117] "RemoveContainer" containerID="af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0"
Sep 10 15:10:30.981521 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:10:30.981495    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0\": container with ID starting with af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0 not found: ID does not exist" containerID="af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0"
Sep 10 15:10:30.981631 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.981531    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0"} err="failed to get container status \"af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0\": rpc error: code = NotFound desc = could not find container \"af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0\": container with ID starting with af50f4065e5d2ff5b1004398c3a9bb6449a3eb5f4b278880e0bd4b852e8a72a0 not found: ID does not exist"
Sep 10 15:10:30.995751 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.995725    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"]
Sep 10 15:10:30.999541 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:30.999520    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ac593d41-kserve-6d8f4c997f-hczzx"]
Sep 10 15:10:31.319296 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:31.319206    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f690402-36c6-40f0-b180-0c88a488b1e3" path="/var/lib/kubelet/pods/6f690402-36c6-40f0-b180-0c88a488b1e3/volumes"
Sep 10 15:10:34.285271 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:34.285233    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:40.399386 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.399348    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:40.644735 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.644698    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"]
Sep 10 15:10:40.645239 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.645211    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f690402-36c6-40f0-b180-0c88a488b1e3" containerName="main"
Sep 10 15:10:40.645239 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.645234    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f690402-36c6-40f0-b180-0c88a488b1e3" containerName="main"
Sep 10 15:10:40.645453 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.645357    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f690402-36c6-40f0-b180-0c88a488b1e3" containerName="main"
Sep 10 15:10:40.649896 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.649841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.661304 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.661260    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"]
Sep 10 15:10:40.779274 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.779237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f70505-56b9-46e4-a526-81e111f38482-maas-api-tls\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.779481 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.779295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f70505-56b9-46e4-a526-81e111f38482-metrics-certs\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.779481 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.779349    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f70505-56b9-46e4-a526-81e111f38482-openshift-service-ca\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.779481 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.779372    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nz7g\" (UniqueName: \"kubernetes.io/projected/f2f70505-56b9-46e4-a526-81e111f38482-kube-api-access-2nz7g\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.880778 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.880738    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f70505-56b9-46e4-a526-81e111f38482-maas-api-tls\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.880778 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.880779    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f70505-56b9-46e4-a526-81e111f38482-metrics-certs\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.881036 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.880806    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f70505-56b9-46e4-a526-81e111f38482-openshift-service-ca\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.881036 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.880827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nz7g\" (UniqueName: \"kubernetes.io/projected/f2f70505-56b9-46e4-a526-81e111f38482-kube-api-access-2nz7g\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.881498 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.881466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f70505-56b9-46e4-a526-81e111f38482-openshift-service-ca\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.883196 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.883170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f70505-56b9-46e4-a526-81e111f38482-maas-api-tls\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.883321 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.883234    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f70505-56b9-46e4-a526-81e111f38482-metrics-certs\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.888062 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.888038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nz7g\" (UniqueName: \"kubernetes.io/projected/f2f70505-56b9-46e4-a526-81e111f38482-kube-api-access-2nz7g\") pod \"maas-api-79b7b5b657-22pw4\" (UID: \"f2f70505-56b9-46e4-a526-81e111f38482\") " pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:40.961397 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:40.961361    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:41.235872 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:41.235832    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"]
Sep 10 15:10:41.236376 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:10:41.236348    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f70505_56b9_46e4_a526_81e111f38482.slice/crio-b6c00da03c231061aa7a6ad45ff903ecc17f128b760b8446a07e271511f732e1 WatchSource:0}: Error finding container b6c00da03c231061aa7a6ad45ff903ecc17f128b760b8446a07e271511f732e1: Status 404 returned error can't find the container with id b6c00da03c231061aa7a6ad45ff903ecc17f128b760b8446a07e271511f732e1
Sep 10 15:10:41.237788 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:41.237770    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:10:42.037612 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:42.037575    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4" event={"ID":"f2f70505-56b9-46e4-a526-81e111f38482","Type":"ContainerStarted","Data":"102a5bf2d0b4e3fe7c04ba770708394ee0c14343098678e0d4e2786086ae8723"}
Sep 10 15:10:42.038026 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:42.037627    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4" event={"ID":"f2f70505-56b9-46e4-a526-81e111f38482","Type":"ContainerStarted","Data":"b6c00da03c231061aa7a6ad45ff903ecc17f128b760b8446a07e271511f732e1"}
Sep 10 15:10:42.038026 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:42.037740    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:42.056663 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:42.056607    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4" podStartSLOduration=1.64769177 podStartE2EDuration="2.056589195s" totalImagesPullingTime="408.897425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:10:42.053818361 +0000 UTC m=+2571.409631297" watchObservedRunningTime="2026-09-10 15:10:42.056589195 +0000 UTC m=+2571.412402103"
Sep 10 15:10:48.050725 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:48.050693    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79b7b5b657-22pw4"
Sep 10 15:10:48.289095 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:48.289059    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:55.511845 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:55.511811    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:10:55.740368 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:55.740331    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"]
Sep 10 15:10:55.740570 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:55.740549    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x" podUID="91f60dbc-7556-4794-96dc-8148f888e5d3" containerName="maas-api" containerID="cri-o://9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3" gracePeriod=30
Sep 10 15:10:56.985759 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:56.985734    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 15:10:57.024183 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.024152    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g729k\" (UniqueName: \"kubernetes.io/projected/91f60dbc-7556-4794-96dc-8148f888e5d3-kube-api-access-g729k\") pod \"91f60dbc-7556-4794-96dc-8148f888e5d3\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") "
Sep 10 15:10:57.024368 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.024202    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f60dbc-7556-4794-96dc-8148f888e5d3-openshift-service-ca\") pod \"91f60dbc-7556-4794-96dc-8148f888e5d3\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") "
Sep 10 15:10:57.024368 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.024250    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") pod \"91f60dbc-7556-4794-96dc-8148f888e5d3\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") "
Sep 10 15:10:57.024490 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.024408    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs\") pod \"91f60dbc-7556-4794-96dc-8148f888e5d3\" (UID: \"91f60dbc-7556-4794-96dc-8148f888e5d3\") "
Sep 10 15:10:57.024610 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.024577    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f60dbc-7556-4794-96dc-8148f888e5d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91f60dbc-7556-4794-96dc-8148f888e5d3" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:10:57.024748 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.024719    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f60dbc-7556-4794-96dc-8148f888e5d3-openshift-service-ca\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:57.026431 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.026406    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91f60dbc-7556-4794-96dc-8148f888e5d3" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:10:57.026753 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.026732    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f60dbc-7556-4794-96dc-8148f888e5d3-kube-api-access-g729k" (OuterVolumeSpecName: "kube-api-access-g729k") pod "91f60dbc-7556-4794-96dc-8148f888e5d3" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3"). InnerVolumeSpecName "kube-api-access-g729k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:10:57.026878 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.026858    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "91f60dbc-7556-4794-96dc-8148f888e5d3" (UID: "91f60dbc-7556-4794-96dc-8148f888e5d3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:10:57.120596 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.120502    2560 generic.go:356] "Generic (PLEG): container finished" podID="91f60dbc-7556-4794-96dc-8148f888e5d3" containerID="9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3" exitCode=0
Sep 10 15:10:57.120596 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.120547    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"
Sep 10 15:10:57.120799 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.120591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x" event={"ID":"91f60dbc-7556-4794-96dc-8148f888e5d3","Type":"ContainerDied","Data":"9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3"}
Sep 10 15:10:57.120799 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.120644    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x" event={"ID":"91f60dbc-7556-4794-96dc-8148f888e5d3","Type":"ContainerDied","Data":"8d23142b7c36ec4516d183c150bac51426c0316bc281908e54d3c7fc9b956216"}
Sep 10 15:10:57.120799 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.120668    2560 scope.go:117] "RemoveContainer" containerID="9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3"
Sep 10 15:10:57.125208 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.125185    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-metrics-certs\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:57.125331 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.125211    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g729k\" (UniqueName: \"kubernetes.io/projected/91f60dbc-7556-4794-96dc-8148f888e5d3-kube-api-access-g729k\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:57.125331 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.125227    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f60dbc-7556-4794-96dc-8148f888e5d3-maas-api-tls\") on node \"ip-10-0-132-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:57.130659 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.130640    2560 scope.go:117] "RemoveContainer" containerID="9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3"
Sep 10 15:10:57.130938 ip-10-0-132-253 kubenswrapper[2560]: E0910 15:10:57.130919    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3\": container with ID starting with 9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3 not found: ID does not exist" containerID="9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3"
Sep 10 15:10:57.130998 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.130948    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3"} err="failed to get container status \"9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3\": rpc error: code = NotFound desc = could not find container \"9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3\": container with ID starting with 9236b389a2abc486a0be9d873015834e5d81b8bee37615011e624c2e06fa7bd3 not found: ID does not exist"
Sep 10 15:10:57.142542 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.142515    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"]
Sep 10 15:10:57.144675 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.144657    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59bbbd559c-pzk5x"]
Sep 10 15:10:57.320127 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:10:57.320088    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f60dbc-7556-4794-96dc-8148f888e5d3" path="/var/lib/kubelet/pods/91f60dbc-7556-4794-96dc-8148f888e5d3/volumes"
Sep 10 15:11:03.889779 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:03.889741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:11:09.210544 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:09.210510    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-v24tf"]
Sep 10 15:11:17.565608 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:17.565519    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qblnw_2888bbca-5f69-43b4-88cf-e3168498706f/manager/0.log"
Sep 10 15:11:17.708627 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:17.708599    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b457c4dbd-45ppx_c4578920-1c2c-4bcd-b188-c55e4a89cbc3/manager/0.log"
Sep 10 15:11:18.091097 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:18.091068    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77dd699645-7lfz4_cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1/manager/0.log"
Sep 10 15:11:18.100152 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:18.100130    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77dd699645-7lfz4_cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1/copy-manifests/0.log"
Sep 10 15:11:18.745644 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:18.745615    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79b7b5b657-22pw4_f2f70505-56b9-46e4-a526-81e111f38482/maas-api/0.log"
Sep 10 15:11:20.159729 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:20.159697    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-c5gtp_72bb8b69-9b7c-4f24-89f7-00124fe570ab/manager/0.log"
Sep 10 15:11:20.282597 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:20.282566    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-2jjlc_5f485167-8a1a-494a-bebb-5a8ec3c8fff2/manager/0.log"
Sep 10 15:11:20.404870 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:20.404840    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-w5lpc_8cdeac31-1b97-4dad-91ae-4e9d95015421/kuadrant-console-plugin/0.log"
Sep 10 15:11:20.663374 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:20.663341    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-v24tf_68ca634a-fcb0-4dda-8e1b-12829c81221a/limitador/0.log"
Sep 10 15:11:20.787951 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:20.787920    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-7qwrx_4e0fa36a-408a-4c5b-92ca-1d01f0273eed/manager/0.log"
Sep 10 15:11:21.486523 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:21.486493    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c68946dcf-spdl2_754e6c11-7151-4277-b955-c56bfc53da50/kube-auth-proxy/0.log"
Sep 10 15:11:21.722247 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:21.722213    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lgfb6_9e1afda4-6e02-4a5b-89e0-206473f32957/payload-pre-processing/0.log"
Sep 10 15:11:21.841044 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:21.840969    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-bzddb_427792ba-b7e1-4f48-957c-feec063e4e34/payload-processing/0.log"
Sep 10 15:11:21.960185 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:21.960154    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758b49888b-qhk2k_8343c732-0f02-49d0-b506-c04b91e1e990/router/0.log"
Sep 10 15:11:22.431981 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:22.431952    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ktfz5_eda000be-a4db-46a6-a7aa-45eacdb38efe/main/0.log"
Sep 10 15:11:22.657021 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:22.656975    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8s2hr_c0a1d7a4-8f14-455a-a16c-1e4a77060d38/main/0.log"
Sep 10 15:11:22.882292 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:22.882259    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8db7v_33f722f0-9774-4876-9f5d-5de63651a8e0/main/0.log"
Sep 10 15:11:22.994014 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:22.993983    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-m7bdb_9cad246b-1533-458f-b51e-4eb04945acd7/main/0.log"
Sep 10 15:11:29.092105 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:29.092022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:11:29.178990 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:29.178958    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:11:32.220984 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:32.220958    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ndntw_807de0e5-36be-4155-b273-fc7eb43f2f53/global-pull-secret-syncer/0.log"
Sep 10 15:11:32.275211 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:32.275182    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log"
Sep 10 15:11:32.386731 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:32.386701    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-253.ec2.internal_424c21ac2ec6f354644b2292aeae7f24/haproxy/0.log"
Sep 10 15:11:37.545635 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:37.545601    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-c5gtp_72bb8b69-9b7c-4f24-89f7-00124fe570ab/manager/0.log"
Sep 10 15:11:37.576464 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:37.576441    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-2jjlc_5f485167-8a1a-494a-bebb-5a8ec3c8fff2/manager/0.log"
Sep 10 15:11:37.604988 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:37.604967    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-w5lpc_8cdeac31-1b97-4dad-91ae-4e9d95015421/kuadrant-console-plugin/0.log"
Sep 10 15:11:37.683099 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:37.683073    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-v24tf_68ca634a-fcb0-4dda-8e1b-12829c81221a/limitador/0.log"
Sep 10 15:11:37.744071 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:37.744046    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-7qwrx_4e0fa36a-408a-4c5b-92ca-1d01f0273eed/manager/0.log"
Sep 10 15:11:38.403492 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:38.403460    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bbmm_a2e559fb-9b78-4509-b544-3410d402bda8/konnectivity-agent/0.log" limit=1024
Sep 10 15:11:39.782382 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:39.782252    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-48llk_e0e62375-71d1-498b-b78e-f52dd722895c/kube-state-metrics/0.log"
Sep 10 15:11:39.801108 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:39.801072    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-48llk_e0e62375-71d1-498b-b78e-f52dd722895c/kube-rbac-proxy-main/0.log"
Sep 10 15:11:39.822075 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:39.822054    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-48llk_e0e62375-71d1-498b-b78e-f52dd722895c/kube-rbac-proxy-self/0.log"
Sep 10 15:11:40.083565 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:40.083475    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsj79_34b9b867-9bf7-4932-a6aa-085cb5e700ec/node-exporter/0.log"
Sep 10 15:11:40.106544 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:40.106512    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsj79_34b9b867-9bf7-4932-a6aa-085cb5e700ec/kube-rbac-proxy/0.log"
Sep 10 15:11:40.128774 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:40.128750    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsj79_34b9b867-9bf7-4932-a6aa-085cb5e700ec/init-textfile/0.log"
Sep 10 15:11:42.824257 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.824213    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"]
Sep 10 15:11:42.824689 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.824670    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f60dbc-7556-4794-96dc-8148f888e5d3" containerName="maas-api"
Sep 10 15:11:42.824740 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.824691    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f60dbc-7556-4794-96dc-8148f888e5d3" containerName="maas-api"
Sep 10 15:11:42.824791 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.824780    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f60dbc-7556-4794-96dc-8148f888e5d3" containerName="maas-api"
Sep 10 15:11:42.827878 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.827862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:42.830207 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.830180    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fqt5c\"/\"openshift-service-ca.crt\""
Sep 10 15:11:42.830207 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.830183    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fqt5c\"/\"kube-root-ca.crt\""
Sep 10 15:11:42.830727 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.830705    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fqt5c\"/\"default-dockercfg-f4hgn\""
Sep 10 15:11:42.839701 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.839677    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"]
Sep 10 15:11:42.934387 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.934351    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-lib-modules\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:42.934559 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.934393    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7r4q\" (UniqueName: \"kubernetes.io/projected/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-kube-api-access-t7r4q\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:42.934559 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.934491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-podres\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:42.934559 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.934526    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-sys\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:42.934667 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:42.934606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-proc\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035437 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-proc\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035605 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035456    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-lib-modules\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035605 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035482    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7r4q\" (UniqueName: \"kubernetes.io/projected/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-kube-api-access-t7r4q\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035605 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-podres\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035605 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-proc\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035605 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-sys\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035835 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035624    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-sys\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035835 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035650    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-lib-modules\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.035835 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.035735    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-podres\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.044255 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.044228    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7r4q\" (UniqueName: \"kubernetes.io/projected/9f5fe0bd-d4d6-465b-bd1f-758991df4ffd-kube-api-access-t7r4q\") pod \"perf-node-gather-daemonset-qsdrx\" (UID: \"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd\") " pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.138953 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.138866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:43.410847 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:43.410820    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"]
Sep 10 15:11:43.412691 ip-10-0-132-253 kubenswrapper[2560]: W0910 15:11:43.412660    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9f5fe0bd_d4d6_465b_bd1f_758991df4ffd.slice/crio-6fe2961ddb9f13db2e61d9eaac2e121ee169e874204e04716f969f0ddbb5f032 WatchSource:0}: Error finding container 6fe2961ddb9f13db2e61d9eaac2e121ee169e874204e04716f969f0ddbb5f032: Status 404 returned error can't find the container with id 6fe2961ddb9f13db2e61d9eaac2e121ee169e874204e04716f969f0ddbb5f032
Sep 10 15:11:44.379382 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:44.379347    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx" event={"ID":"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd","Type":"ContainerStarted","Data":"6a15c2e835229093dc0053067459f5dae7a00d35b26fbbcb744356c5f8968f52"}
Sep 10 15:11:44.379775 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:44.379395    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx" event={"ID":"9f5fe0bd-d4d6-465b-bd1f-758991df4ffd","Type":"ContainerStarted","Data":"6fe2961ddb9f13db2e61d9eaac2e121ee169e874204e04716f969f0ddbb5f032"}
Sep 10 15:11:44.379775 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:44.379447    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:44.396084 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:44.396041    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx" podStartSLOduration=2.396027687 podStartE2EDuration="2.396027687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:11:44.39471841 +0000 UTC m=+2633.750531343" watchObservedRunningTime="2026-09-10 15:11:44.396027687 +0000 UTC m=+2633.751840635"
Sep 10 15:11:45.297165 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:45.297134    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-78xpw_c7c25814-e802-4f67-95cd-12a0a7fcd941/dns/0.log"
Sep 10 15:11:45.321521 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:45.321480    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-78xpw_c7c25814-e802-4f67-95cd-12a0a7fcd941/kube-rbac-proxy/0.log"
Sep 10 15:11:45.458185 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:45.458151    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwfht_8d7361c0-0247-4988-bd32-fda7907cb325/dns-node-resolver/0.log"
Sep 10 15:11:46.075801 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:46.075768    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pksmv_6cddba93-d2fc-4e91-be06-a8559e276d53/node-ca/0.log"
Sep 10 15:11:47.353904 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:47.353873    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c68946dcf-spdl2_754e6c11-7151-4277-b955-c56bfc53da50/kube-auth-proxy/0.log"
Sep 10 15:11:47.428382 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:47.428352    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lgfb6_9e1afda4-6e02-4a5b-89e0-206473f32957/payload-pre-processing/0.log"
Sep 10 15:11:47.467205 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:47.467178    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-bzddb_427792ba-b7e1-4f48-957c-feec063e4e34/payload-processing/0.log"
Sep 10 15:11:47.501058 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:47.501035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758b49888b-qhk2k_8343c732-0f02-49d0-b506-c04b91e1e990/router/0.log"
Sep 10 15:11:48.533192 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:48.533160    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6sxkb_e4e1db6f-fa90-47e4-86eb-23713483bc05/serve-healthcheck-canary/0.log"
Sep 10 15:11:49.074693 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:49.074659    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-5sv6n_1d2a0a89-de95-498a-b00e-7ad8ecc0e906/insights-operator/0.log"
Sep 10 15:11:49.075056 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:49.075039    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-5sv6n_1d2a0a89-de95-498a-b00e-7ad8ecc0e906/insights-operator/1.log"
Sep 10 15:11:49.096328 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:49.096296    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6j84z_8df946e7-5aad-467f-a2bb-0e19d6f8ee50/kube-rbac-proxy/0.log"
Sep 10 15:11:49.116122 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:49.116104    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6j84z_8df946e7-5aad-467f-a2bb-0e19d6f8ee50/exporter/0.log"
Sep 10 15:11:49.140537 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:49.140512    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6j84z_8df946e7-5aad-467f-a2bb-0e19d6f8ee50/extractor/0.log"
Sep 10 15:11:50.395134 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:50.395095    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fqt5c/perf-node-gather-daemonset-qsdrx"
Sep 10 15:11:51.709935 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:51.709901    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qblnw_2888bbca-5f69-43b4-88cf-e3168498706f/manager/0.log"
Sep 10 15:11:51.760914 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:51.760887    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b457c4dbd-45ppx_c4578920-1c2c-4bcd-b188-c55e4a89cbc3/manager/0.log"
Sep 10 15:11:51.923545 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:51.923515    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77dd699645-7lfz4_cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1/manager/0.log"
Sep 10 15:11:51.949639 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:51.949615    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77dd699645-7lfz4_cb9c4207-0c89-48fb-bc01-4b8f3cf30bb1/copy-manifests/0.log"
Sep 10 15:11:53.864438 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:53.864362    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c68946dcf-spdl2_754e6c11-7151-4277-b955-c56bfc53da50/kube-auth-proxy/0.log"
Sep 10 15:11:53.914990 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:53.914961    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lgfb6_9e1afda4-6e02-4a5b-89e0-206473f32957/payload-pre-processing/0.log"
Sep 10 15:11:53.943119 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:53.943072    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-bzddb_427792ba-b7e1-4f48-957c-feec063e4e34/payload-processing/0.log"
Sep 10 15:11:53.966098 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:53.966069    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758b49888b-qhk2k_8343c732-0f02-49d0-b506-c04b91e1e990/router/0.log"
Sep 10 15:11:55.852835 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:55.852800    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c68946dcf-spdl2_754e6c11-7151-4277-b955-c56bfc53da50/kube-auth-proxy/0.log"
Sep 10 15:11:55.895881 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:55.895846    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lgfb6_9e1afda4-6e02-4a5b-89e0-206473f32957/payload-pre-processing/0.log"
Sep 10 15:11:55.919552 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:55.919521    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-bzddb_427792ba-b7e1-4f48-957c-feec063e4e34/payload-processing/0.log"
Sep 10 15:11:55.937064 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:55.937035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758b49888b-qhk2k_8343c732-0f02-49d0-b506-c04b91e1e990/router/0.log"
Sep 10 15:11:59.013200 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:59.013166    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-6dpbx_1bc46da6-b07e-4f27-b980-1acad1ba60b0/migrator/0.log"
Sep 10 15:11:59.032594 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:11:59.032570    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-6dpbx_1bc46da6-b07e-4f27-b980-1acad1ba60b0/graceful-termination/0.log"
Sep 10 15:12:00.468806 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.468775    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/kube-multus-additional-cni-plugins/0.log"
Sep 10 15:12:00.491396 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.491368    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/egress-router-binary-copy/0.log"
Sep 10 15:12:00.513288 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.513259    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/cni-plugins/0.log"
Sep 10 15:12:00.534323 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.534297    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/bond-cni-plugin/0.log"
Sep 10 15:12:00.554132 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.554108    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/routeoverride-cni/0.log"
Sep 10 15:12:00.576146 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.576123    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/whereabouts-cni-bincopy/0.log"
Sep 10 15:12:00.599161 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.599141    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bhrh_53311511-e278-4666-bb1f-30c6c101fede/whereabouts-cni/0.log"
Sep 10 15:12:00.970648 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:00.970616    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d8sfz_2db006d1-41d7-487e-a3ee-ded18e3db48a/kube-multus/0.log"
Sep 10 15:12:01.027151 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:01.027116    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-57lnd_d0a7d218-c488-478d-ad91-3e119f318ad0/network-metrics-daemon/0.log"
Sep 10 15:12:01.044670 ip-10-0-132-253 kubenswrapper[2560]: I0910 15:12:01.044645    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-57lnd_d0a7d218-c488-478d-ad91-3e119f318ad0/kube-rbac-proxy/0.log"